Showing 1–50 of 168 results for author: Xing, C

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing

Authors: Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Peng Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Ranran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Jiayang Cheng, Zhaowei Wang, Ying Su, Raj Sanjay Shah, Ruohao Guo , et al. (15 additional authors not shown)

Abstract: This work is motivated by two key trends. On one hand, large language models (LLMs) have shown remarkable versatility in various generative tasks such as writing, drawing, and question answering, significantly reducing the time required for many routine tasks. On the other hand, researchers, whose work is not only time-consuming but also highly expertise-demanding, face increasing challenges as th… ▽ More This work is motivated by two key trends. On one hand, large language models (LLMs) have shown remarkable versatility in various generative tasks such as writing, drawing, and question answering, significantly reducing the time required for many routine tasks. On the other hand, researchers, whose work is not only time-consuming but also highly expertise-demanding, face increasing challenges as they have to spend more time reading, writing, and reviewing papers. This raises the question: how can LLMs potentially assist researchers in alleviating their heavy workload? This study focuses on the topic of LLMs assist NLP Researchers, particularly examining the effectiveness of LLM in assisting paper (meta-)reviewing and its recognizability. To address this, we constructed the ReviewCritique dataset, which includes two types of information: (i) NLP papers (initial submissions rather than camera-ready) with both human-written and LLM-generated reviews, and (ii) each review comes with "deficiency" labels and corresponding explanations for individual segments, annotated by experts. Using ReviewCritique, this study explores two threads of research questions: (i) "LLMs as Reviewers", how do reviews generated by LLMs compare with those written by humans in terms of quality and distinguishability? (ii) "LLMs as Metareviewers", how effectively can LLMs identify potential issues, such as Deficient or unprofessional review segments, within individual paper reviews? To our knowledge, this is the first work to provide such a comprehensive analysis. △ Less

Submitted 25 June, 2024; v1 submitted 23 June, 2024; originally announced June 2024.

Harnessing GPU Power for Enhanced OLTP: A Study in Concurrency Control Schemes

Authors: Zihan Sun, Yong Zhang, Chao Li, Chunxiao Xing

Abstract: GPUs, whose performance has gone through a huge leap over the past decade, have proved their ability to accelerate Online Analytical Processing (OLAP) operations. On the other hand, there is still a huge gap in the field of GPU-accelerated Online Transaction Processing (OLTP) operations since it was generally believed that GPUswere not suitable for OLTP in the past. However, the massive parallelis… ▽ More GPUs, whose performance has gone through a huge leap over the past decade, have proved their ability to accelerate Online Analytical Processing (OLAP) operations. On the other hand, there is still a huge gap in the field of GPU-accelerated Online Transaction Processing (OLTP) operations since it was generally believed that GPUswere not suitable for OLTP in the past. However, the massive parallelism and high memory bandwidth give GPUs the potential to process thousands of transactions concurrently. Among the components of OLTP systems, Concurrency Control (CC) schemes have a great impact on the performance of transaction processing and they may behave differently on GPUs because of the different hardware architectures between GPUs and CPUs. In this paper, we design and build the first test-bed gCCTB for CCschemes on GPUsandimplement eight CC schemes for gCCTB. These schemes include six common schemes previously designed for CPUs and two schemes designed for GPUs. Then we make a comprehensive evaluation of these CC schemes with YCSB and TPC-C benchmarks and a number of launch parameters on GPUs. The experience accumulated on our test-bed can assist researchers andengineers to design andimplementnewGPU-acceleratedOLTP systems. Furthermore, the results of our evaluation cast light on research directions of high performance CC schemes on GPUs. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Navigation and 3D Surface Reconstruction from Passive Whisker Sensing

Authors: Michael A. Lin, Hao Li, Chengyi Xing, Mark R. Cutkosky

Abstract: Whiskers provide a way to sense surfaces in the immediate environment without disturbing it. In this paper we present a method for using highly flexible, curved, passive whiskers mounted along a robot arm to gather sensory data as they brush past objects during normal robot motion. The information is useful both for guiding the robot in cluttered spaces and for reconstructing the exposed faces of… ▽ More Whiskers provide a way to sense surfaces in the immediate environment without disturbing it. In this paper we present a method for using highly flexible, curved, passive whiskers mounted along a robot arm to gather sensory data as they brush past objects during normal robot motion. The information is useful both for guiding the robot in cluttered spaces and for reconstructing the exposed faces of objects. Surface reconstruction depends on accurate localization of contact points along each whisker. We present an algorithm based on Bayesian filtering that rapidly converges to within 1\,mm of the actual contact locations. The piecewise-continuous history of contact locations from each whisker allows for accurate reconstruction of curves on object surfaces. Employing multiple whiskers and traces, we are able to produce an occupancy map of proximal objects. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: arXiv admin note: text overlap with arXiv:2210.12387

MatrixGate: A High-performance Data Ingestion Tool for Time-series Databases

Authors: Shuhui Wang, Zihan Sun, Chaochen Hu, Chao Li, Yong Zhang, Yandong Yao, Hao Wang, Chunxiao Xing

Abstract: Recent years have seen massive time-series data generated in many areas. This different scenario brings new challenges, particularly in terms of data ingestion, where existing technologies struggle to handle such massive time-series data, leading to low loading speed and poor timeliness. To address these challenges, this paper presents MatrixGate, a new and efficient data ingestion approach for ma… ▽ More Recent years have seen massive time-series data generated in many areas. This different scenario brings new challenges, particularly in terms of data ingestion, where existing technologies struggle to handle such massive time-series data, leading to low loading speed and poor timeliness. To address these challenges, this paper presents MatrixGate, a new and efficient data ingestion approach for massive time-series data. MatrixGate implements both single-instance and multi-instance parallel procedures, which is based on its unique ingestion strategies. First, MatrixGate uses policies to tune the slots that are synchronized with segments to ingest data, which eliminates the cost of starting transactions and enhance the efficiency. Second, multi-coroutines are responsible for transfer data, which can increase the degree of parallelism significantly. Third, lock-free queues are used to enable direct data transfer without the need for disk storage or lodging in the master instance. Experiment results on multiple datasets show that MatrixGate outperforms state-of-the-art methods by 3 to 100 times in loading speed, and cuts down about 80% query latency. Furthermore, MatrixGate scales out efficiently under distributed architecture, achieving scalability of 86%. △ Less

Submitted 8 June, 2024; originally announced June 2024.

Grasp as You Say: Language-guided Dexterous Grasp Generation

Authors: Yi-Lin Wei, Jian-Jian Jiang, Chengyi Xing, Xiantuo Tan, Xiao-Ming Wu, Hao Li, Mark Cutkosky, Wei-Shi Zheng

Abstract: This paper explores a novel task ""Dexterous Grasp as You Say"" (DexGYS), enabling robots to perform dexterous gras** based on human commands expressed in natural language. However, the development of this field is hindered by the lack of datasets with natural human guidance; thus, we propose a language-guided dexterous grasp dataset, named DexGYSNet, offering high-quality dexterous grasp annota… ▽ More This paper explores a novel task ""Dexterous Grasp as You Say"" (DexGYS), enabling robots to perform dexterous gras** based on human commands expressed in natural language. However, the development of this field is hindered by the lack of datasets with natural human guidance; thus, we propose a language-guided dexterous grasp dataset, named DexGYSNet, offering high-quality dexterous grasp annotations along with flexible and fine-grained human language guidance. Our dataset construction is cost-efficient, with the carefully-design hand-object interaction retargeting strategy, and the LLM-assisted language guidance annotation system. Equipped with this dataset, we introduce the DexGYSGrasp framework for generating dexterous grasps based on human language instructions, with the capability of producing grasps that are intent-aligned, high quality and diversity. To achieve this capability, our framework decomposes the complex learning process into two manageable progressive objectives and introduce two components to realize them. The first component learns the grasp distribution focusing on intention alignment and generation diversity. And the second component refines the grasp quality while maintaining intention consistency. Extensive experiments are conducted on DexGYSNet and real world environment for validation. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 9 pages, 7 figures

Single-View Scene Point Cloud Human Grasp Generation

Authors: Yan-Kang Wang, Chengyi Xing, Yi-Lin Wei, Xiao-Ming Wu, Wei-Shi Zheng

Abstract: In this work, we explore a novel task of generating human grasps based on single-view scene point clouds, which more accurately mirrors the typical real-world situation of observing objects from a single viewpoint. Due to the incompleteness of object point clouds and the presence of numerous scene points, the generated hand is prone to penetrating into the invisible parts of the object and the mod… ▽ More In this work, we explore a novel task of generating human grasps based on single-view scene point clouds, which more accurately mirrors the typical real-world situation of observing objects from a single viewpoint. Due to the incompleteness of object point clouds and the presence of numerous scene points, the generated hand is prone to penetrating into the invisible parts of the object and the model is easily affected by scene points. Thus, we introduce S2HGrasp, a framework composed of two key modules: the Global Perception module that globally perceives partial object point clouds, and the DiffuGrasp module designed to generate high-quality human grasps based on complex inputs that include scene points. Additionally, we introduce S2HGD dataset, which comprises approximately 99,000 single-object single-view scene point clouds of 1,668 unique objects, each annotated with one human grasp. Our extensive experiments demonstrate that S2HGrasp can not only generate natural human grasps regardless of scene points, but also effectively prevent penetration between the hand and invisible parts of the object. Moreover, our model showcases strong generalization capability when applied to unseen objects. Our code and dataset are available at https://github.com/iSEE-Laboratory/S2HGrasp. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Random Gabidulin Codes Achieve List Decoding Capacity in the Rank Metric

Authors: Zeyu Guo, Chen Yuan, Zihan Zhang

Abstract: Gabidulin codes, serving as the rank-metric counterpart of Reed-Solomon codes, constitute an important class of maximum rank distance (MRD) codes. However, unlike the fruitful positive results about the list decoding of Reed-Solomon codes, results concerning the list decodability of Gabidulin codes in the rank metric are all negative so far. For example, in contrast to Reed-Solomon codes, which ar… ▽ More Gabidulin codes, serving as the rank-metric counterpart of Reed-Solomon codes, constitute an important class of maximum rank distance (MRD) codes. However, unlike the fruitful positive results about the list decoding of Reed-Solomon codes, results concerning the list decodability of Gabidulin codes in the rank metric are all negative so far. For example, in contrast to Reed-Solomon codes, which are always list decodable up to the Johnson bound in the Hamming metric, Raviv and Wachter-Zeh (IEEE TIT, 2016 and 2017) constructed a class of Gabidulin codes that are not even combinatorially list decodable beyond the unique decoding radius in the rank metric. Proving the existence of Gabidulin codes with good combinatorial list decodability in the rank metric has remained a long-standing open problem. In this paper, we resolve the aforementioned open problem by showing that, with high probability, random Gabidulin codes over sufficiently large alphabets attain the optimal generalized Singleton bound for list decoding in the rank metric. In particular, they achieve list decoding capacity in the rank metric. Our work is significantly influenced by the recent breakthroughs in the combinatorial list decodability of Reed-Solomon codes, especially the work by Brakensiek, Gopi, and Makam (STOC 2023). Our major technical contributions, which may hold independent interest, consist of the following: (1) We initiate the study of ``higher order MRD codes'' and provide a novel unified theory, which runs parallel to the theory of ``higher order MDS codes'' developed by BGM. (2) We prove a natural analog of the GM-MDS theorem, proven by Lovett (FOCS 2018) and Yildiz and Hassibi (IEEE TIT, 2019), which we call the GM-MRD theorem. In particular, our GM-MRD theorem for Gabidulin codes are strictly stronger than the GM-MDS theorem for Gabidulin codes, proven by Yildiz and Hassibi (IEEE TIT, 2019). △ Less

Submitted 19 April, 2024; originally announced April 2024.

FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability

Authors: Congying Xia, Chen Xing, Jiangshu Du, Xinyi Yang, Yihao Feng, Ran Xu, Wenpeng Yin, Caiming Xiong

Abstract: This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats, a crucial yet underexamined capability for their application as AI agents. Despite LLMs' advancements, existing benchmarks fail to assess their format-following proficiency adequately. FoFo fills this gap with a diverse range of real-world formats and in… ▽ More This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats, a crucial yet underexamined capability for their application as AI agents. Despite LLMs' advancements, existing benchmarks fail to assess their format-following proficiency adequately. FoFo fills this gap with a diverse range of real-world formats and instructions, developed through an AI-Human collaborative method. Our evaluation across both open-source (e.g., Llama 2, WizardLM) and closed-source (e.g., GPT-4, PALM2, Gemini) LLMs highlights three key findings: open-source models significantly lag behind closed-source ones in format adherence; LLMs' format-following performance is independent of their content generation quality; and LLMs' format proficiency varies across different domains. These insights suggest the need for specialized tuning for format-following skills and highlight FoFo's role in guiding the selection of domain-specific AI agents. FoFo is released here at https://github.com/SalesforceAIResearch/FoFo. △ Less

Submitted 28 February, 2024; originally announced February 2024.

Comments: The first two authors contributed equally

Asymptotic construction of locally repairable codes with multiple recovering sets

Authors: Singsong Li, Shu Liu, Liming Ma, Abstract: Locally repairable codes have been extensively investigated due to practical applications in distributed and cloud storage systems in recent years. However, not much work on asymptotic behavior of locally repairable codes has been done. In particular, there is few result on constructive lower bound of asymptotic behavior of locally repairable codes with multiple recovering sets. In this paper, we… ▽ More Locally repairable codes have been extensively investigated due to practical applications in distributed and cloud storage systems in recent years. However, not much work on asymptotic behavior of locally repairable codes has been done. In particular, there is few result on constructive lower bound of asymptotic behavior of locally repairable codes with multiple recovering sets. In this paper, we construct some families of asymptotically good locally repairable codes with multiple recovering sets via automorphism groups of function fields of the Garcia-Stichtenoth towers. The main advantage of our construction is to allow more flexibility of localities. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Massive Unsourced Random Access for Near-Field Communications

Authors: Xinyu Xie, Yongpeng Wu, Jian** An, Derrick Wing Kwan Ng, Chengwen Xing, Wenjun Zhang

Abstract: This paper investigates the unsourced random access (URA) problem with a massive multiple-input multiple-output receiver that serves wireless devices in the near-field of radiation. We employ an uncoupled transmission protocol without appending redundancies to the slot-wise encoded messages. To exploit the channel sparsity for block length reduction while facing the collapsed sparse structure in t… ▽ More This paper investigates the unsourced random access (URA) problem with a massive multiple-input multiple-output receiver that serves wireless devices in the near-field of radiation. We employ an uncoupled transmission protocol without appending redundancies to the slot-wise encoded messages. To exploit the channel sparsity for block length reduction while facing the collapsed sparse structure in the angular domain of near-field channels, we propose a sparse channel sampling method that divides the angle-distance (polar) domain based on the maximum permissible coherence. Decoding starts with retrieving active codewords and channels from each slot. We address the issue by leveraging the structured channel sparsity in the spatial and polar domains and propose a novel turbo-based recovery algorithm. Furthermore, we investigate an off-grid compressed sensing method to refine discretely estimated channel parameters over the continuum that improves the detection performance. Afterward, without the assistance of redundancies, we recouple the separated messages according to the similarity of the users' channel information and propose a modified K-medoids method to handle the constraints and collisions involved in channel clustering. Simulations reveal that via exploiting the channel sparsity, the proposed URA scheme achieves high spectral efficiency and surpasses existing multi-slot-based schemes. Moreover, with more measurements provided by the overcomplete channel sampling, the near-field-suited scheme outperforms its counterpart of the far-field. △ Less

Submitted 25 January, 2024; originally announced January 2024.

Comments: Accepted by IEEE Transactions on Communications

Investigating Inter-Satellite Link Spanning Patterns on Networking Performance in Mega-constellations

Authors: Xiangtong Wang, Xiaodong Han, Menglong Yang, Chuan Xing, Yuqi Wang, Songchen Han, Wei Li

Abstract: Low Earth orbit (LEO) mega-constellations rely on inter-satellite links (ISLs) to provide global connectivity. We note that in addition to the general constellation parameters, the ISL spanning patterns are also greatly influence the final network structure and thus the network performance. In this work, we formulate the ISL spanning patterns, apply different patterns to mega-constellation and g… ▽ More Low Earth orbit (LEO) mega-constellations rely on inter-satellite links (ISLs) to provide global connectivity. We note that in addition to the general constellation parameters, the ISL spanning patterns are also greatly influence the final network structure and thus the network performance. In this work, we formulate the ISL spanning patterns, apply different patterns to mega-constellation and generate multiple structures. Then, we delve into the performance estimation of these networks, specifically evaluating network capacity, throughput, latency, and routing path stretch. The experimental findings provide insights into the optimal network structure under diverse conditions, showcasing superior performance when compared to alternative network configurations. △ Less

Submitted 25 December, 2023; originally announced December 2023.

Comments: 5pages

E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation

Authors: Xinhang Li, Chong Chen, Xiangyu Zhao, Yong Zhang, Chunxiao Xing

Abstract: The recent advancements in Large Language Models (LLMs) have sparked interest in harnessing their potential within recommender systems. Since LLMs are designed for natural language tasks, existing recommendation approaches have predominantly transformed recommendation tasks into open-domain natural language generation tasks. However, this approach necessitates items to possess rich semantic inform… ▽ More The recent advancements in Large Language Models (LLMs) have sparked interest in harnessing their potential within recommender systems. Since LLMs are designed for natural language tasks, existing recommendation approaches have predominantly transformed recommendation tasks into open-domain natural language generation tasks. However, this approach necessitates items to possess rich semantic information, often generates out-of-range results, and suffers from notably low efficiency and limited extensibility. Furthermore, practical ID-based recommendation strategies, reliant on a huge number of unique identities (IDs) to represent users and items, have gained prominence in real-world recommender systems due to their effectiveness and efficiency. Nevertheless, the incapacity of LLMs to model IDs presents a formidable challenge when seeking to leverage LLMs for personalized recommendations. In this paper, we introduce an Elegant Effective Efficient Extensible solution for large language models for Sequential Recommendation (E4SRec), which seamlessly integrates LLMs with traditional recommender systems that exclusively utilize IDs to represent items. Specifically, E4SRec takes ID sequences as inputs, ensuring that the generated outputs fall within the candidate lists. Furthermore, E4SRec possesses the capability to generate the entire ranking list in a single forward process, and demands only a minimal set of pluggable parameters, which are trained for each dataset while kee** the entire LLM frozen. We substantiate the effectiveness, efficiency, and extensibility of our proposed E4SRec through comprehensive experiments conducted on four widely-used real-world datasets. The implementation code is accessible at https://github.com/HestiaSky/E4SRec/. △ Less

Submitted 4 December, 2023; originally announced December 2023.

A Framework on Complex Matrix Derivatives with Special Structure Constraints for Wireless Systems

Authors: Xin Ju, Shiqi Gong, Nan Zhao, Chengwen Xing, Arumugam Nallanathan, Dusit Niyato

Abstract: Matrix-variate optimization plays a central role in advanced wireless system designs. In this paper, we aim to explore optimal solutions of matrix variables under two special structure constraints using complex matrix derivatives, including diagonal structure constraints and constant modulus constraints, both of which are closely related to the state-of-the-art wireless applications. Specifically,… ▽ More Matrix-variate optimization plays a central role in advanced wireless system designs. In this paper, we aim to explore optimal solutions of matrix variables under two special structure constraints using complex matrix derivatives, including diagonal structure constraints and constant modulus constraints, both of which are closely related to the state-of-the-art wireless applications. Specifically, for diagonal structure constraints mostly considered in the uplink multi-user single-input multiple-output (MU-SIMO) system and the amplitude-adjustable intelligent reflecting surface (IRS)-aided multiple-input multiple-output (MIMO) system, the capacity maximization problem, the mean-squared error (MSE) minimization problem and their variants are rigorously investigated. By leveraging complex matrix derivatives, the optimal solutions of these problems are directly obtained in closed forms. Nevertheless, for constant modulus constraints with the intrinsic nature of element-wise decomposability, which are often seen in the hybrid analog-digital MIMO system and the fully-passive IRS-aided MIMO system, we firstly explore inherent structures of the element-wise phase derivatives associated with different optimization problems. Then, we propose a novel alternating optimization (AO) algorithm with the aid of several arbitrary feasible solutions, which avoids the complicated matrix inversion and matrix factorization involved in conventional element-wise iterative algorithms. Numerical simulations reveal that the proposed algorithm can dramatically reduce the computational complexity without loss of system performance. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Fast Fourier transform via automorphism groups of rational function fields

Authors: Songsong Li, Abstract: The Fast Fourier Transform (FFT) over a finite field $\mathbb{F}_q$ computes evaluations of a given polynomial of degree less than $n$ at a specifically chosen set of $n$ distinct evaluation points in $\mathbb{F}_q$. If $q$ or $q-1$ is a smooth number, then the divide-and-conquer approach leads to the fastest known FFT algorithms. Depending on the type of group that the set of evaluation points fo… ▽ More The Fast Fourier Transform (FFT) over a finite field $\mathbb{F}_q$ computes evaluations of a given polynomial of degree less than $n$ at a specifically chosen set of $n$ distinct evaluation points in $\mathbb{F}_q$. If $q$ or $q-1$ is a smooth number, then the divide-and-conquer approach leads to the fastest known FFT algorithms. Depending on the type of group that the set of evaluation points forms, these algorithms are classified as multiplicative (Math of Comp. 1965) and additive (FOCS 2014) FFT algorithms. In this work, we provide a unified framework for FFT algorithms that include both multiplicative and additive FFT algorithms as special cases, and beyond: our framework also works when $q+1$ is smooth, while all known results require $q$ or $q-1$ to be smooth. For the new case where $q+1$ is smooth (this new case was not considered before in literature as far as we know), we show that if $n$ is a divisor of $q+1$ that is $B$-smooth for a real $B>0$, then our FFT needs $O(Bn\log n)$ arithmetic operations in $\mathbb{F}_q$. Our unified framework is a natural consequence of introducing the algebraic function fields into the study of FFT. △ Less

Submitted 22 October, 2023; originally announced October 2023.

Nonlinear Codes with Low Redundancy

Authors: Shu Liu, Abstract: Determining the largest size, or equivalently finding the lowest redundancy, of q-ary codes for given length and minimum distance is one of the central and fundamental problems in coding theory. Inspired by the construction of Varshamov-Tenengolts (VT for short) codes via check-sums, we provide an explicit construction of nonlinear codes with lower redundancy than linear codes under the same lengt… ▽ More Determining the largest size, or equivalently finding the lowest redundancy, of q-ary codes for given length and minimum distance is one of the central and fundamental problems in coding theory. Inspired by the construction of Varshamov-Tenengolts (VT for short) codes via check-sums, we provide an explicit construction of nonlinear codes with lower redundancy than linear codes under the same length and minimum distance. Similar to the VT codes, our construction works well for small distance (or even constant distance). Furthermore, we design quasi-linear time decoding algorithms for both erasure and adversary errors. △ Less

Submitted 22 October, 2023; originally announced October 2023.

Lemur: Harmonizing Natural Language and Code for Language Agents

Authors: Yiheng Xu, Hong** Su, Chen Xing, Boyu Mi, Qian Liu, Weijia Shi, Binyuan Hui, Fan Zhou, Yitao Liu, Tianbao Xie, Zhoujun Cheng, Siheng Zhao, Lingpeng Kong, Bailin Wang, Caiming Xiong, Tao Yu

Abstract: We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents. The evolution from language chat models to functional language agents demands that models not only master human interaction, reasoning, and planning but also ensure grounding in the relevant environments. This calls… ▽ More We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents. The evolution from language chat models to functional language agents demands that models not only master human interaction, reasoning, and planning but also ensure grounding in the relevant environments. This calls for a harmonious blend of language and coding capabilities in the models. Lemur and Lemur-Chat are proposed to address this necessity, demonstrating balanced proficiencies in both domains, unlike existing open-source models that tend to specialize in either. Through meticulous pre-training using a code-intensive corpus and instruction fine-tuning on text and code data, our models achieve state-of-the-art averaged performance across diverse text and coding benchmarks among open-source models. Comprehensive experiments demonstrate Lemur's superiority over existing open-source models and its proficiency across various agent tasks involving human communication, tool usage, and interaction under fully- and partially- observable environments. The harmonization between natural and programming languages enables Lemur-Chat to significantly narrow the gap with proprietary models on agent abilities, providing key insights into develo** advanced open-source agents adept at reasoning, planning, and operating seamlessly across environments. https://github.com/OpenLemur/Lemur △ Less

Submitted 10 October, 2023; originally announced October 2023.

Scene-aware Human Motion Forecasting via Mutual Distance Prediction

Authors: Chaoyue Xing, Wei Mao, Miaomiao Liu

Abstract: In this paper, we tackle the problem of scene-aware 3D human motion forecasting. A key challenge of this task is to predict future human motions that are consistent with the scene by modeling the human-scene interactions. While recent works have demonstrated that explicit constraints on human-scene interactions can prevent the occurrence of ghost motion, they only provide constraints on partial hu… ▽ More In this paper, we tackle the problem of scene-aware 3D human motion forecasting. A key challenge of this task is to predict future human motions that are consistent with the scene by modeling the human-scene interactions. While recent works have demonstrated that explicit constraints on human-scene interactions can prevent the occurrence of ghost motion, they only provide constraints on partial human motion e.g., the global motion of the human or a few joints contacting the scene, leaving the rest of the motion unconstrained. To address this limitation, we propose to model the human-scene interaction with the mutual distance between the human body and the scene. Such mutual distances constrain both the local and global human motion, resulting in a whole-body motion constrained prediction. In particular, mutual distance constraints consist of two components, the signed distance of each vertex on the human mesh to the scene surface and the distance of basis scene points to the human mesh. We further introduce a global scene representation learned from a signed distance function (SDF) volume to ensure coherence between the global scene representation and the explicit constraint from the mutual distance. We develop a pipeline with two sequential steps: predicting the future mutual distances first, followed by forecasting future human motion. During training, we explicitly encourage consistency between predicted poses and mutual distances. Extensive evaluations on the existing synthetic and real datasets demonstrate that our approach consistently outperforms the state-of-the-art methods. △ Less

Submitted 4 April, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight

Authors: Jiaxu Xing, Leonard Bauersfeld, Yunlong Song, Chunwei Xing, Davide Scaramuzza

Abstract: Scene transfer for vision-based mobile robotics applications is a highly relevant and challenging problem. The utility of a robot greatly depends on its ability to perform a task in the real world, outside of a well-controlled lab environment. Existing scene transfer end-to-end policy learning approaches often suffer from poor sample efficiency or limited generalization capabilities, making them u… ▽ More Scene transfer for vision-based mobile robotics applications is a highly relevant and challenging problem. The utility of a robot greatly depends on its ability to perform a task in the real world, outside of a well-controlled lab environment. Existing scene transfer end-to-end policy learning approaches often suffer from poor sample efficiency or limited generalization capabilities, making them unsuitable for mobile robotics applications. This work proposes an adaptive multi-pair contrastive learning strategy for visual representation learning that enables zero-shot scene transfer and real-world deployment. Control policies relying on the embedding are able to operate in unseen environments without the need for finetuning in the deployment environment. We demonstrate the performance of our approach on the task of agile, vision-based quadrotor flight. Extensive simulation and real-world experiments demonstrate that our approach successfully generalizes beyond the training domain and outperforms all baselines. △ Less

Submitted 29 February, 2024; v1 submitted 18 September, 2023; originally announced September 2023.

Journal ref: IEEE International Conference on Robotics and Automation (ICRA), 2024

XGen-7B Technical Report

Authors: Erik Nijkamp, Tian Xie, Hiroaki Hayashi, Bo Pang, Congying Xia, Chen Xing, Jesse Vig, Semih Yavuz, Philippe Laban, Ben Krause, Senthil Purushwalkam, Tong Niu, Wojciech Kryściński, Lidiya Murakhovs'ka, Prafulla Kumar Choubey, Alex Fabbri, Ye Liu, Rui Meng, Lifu Tu, Meghana Bhat, Chien-Sheng Wu, Silvio Savarese, Yingbo Zhou, Shafiq Joty, Caiming Xiong

Abstract: Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LLMs remain confined behind proprietary walls, hindering scientific progress. Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many t… ▽ More Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LLMs remain confined behind proprietary walls, hindering scientific progress. Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context. To address this, we have trained XGen, a series of 7B parameter models on up to 8K sequence length for up to 1.5T tokens. We have also finetuned the XGen models on public-domain instructional data, creating their instruction-tuned counterparts (XGen-Inst). We open-source our models for both research advancements and commercial applications. Our evaluation on standard benchmarks shows that XGen models achieve comparable or better results when compared with state-of-the-art open-source LLMs. Our targeted evaluation on long sequence modeling tasks shows the benefits of our 8K-sequence models over 2K-sequence open-source LLMs. △ Less

Submitted 6 September, 2023; originally announced September 2023.

P2C: Self-Supervised Point Cloud Completion from Single Partial Clouds

Authors: Ruikai Cui, Shi Qiu, Saeed Anwar, Jiawei Liu, Chaoyue Xing, **g Zhang, Nick Barnes

Abstract: Point cloud completion aims to recover the complete shape based on a partial observation. Existing methods require either complete point clouds or multiple partial observations of the same object for learning. In contrast to previous approaches, we present Partial2Complete (P2C), the first self-supervised framework that completes point cloud objects using training samples consisting of only a sing… ▽ More Point cloud completion aims to recover the complete shape based on a partial observation. Existing methods require either complete point clouds or multiple partial observations of the same object for learning. In contrast to previous approaches, we present Partial2Complete (P2C), the first self-supervised framework that completes point cloud objects using training samples consisting of only a single incomplete point cloud per object. Specifically, our framework groups incomplete point clouds into local patches as input and predicts masked patches by learning prior information from different partial objects. We also propose Region-Aware Chamfer Distance to regularize shape mismatch without limiting completion capability, and devise the Normal Consistency Constraint to incorporate a local planarity assumption, encouraging the recovered shape surface to be continuous and complete. In this way, P2C no longer needs multiple observations or complete point clouds as ground truth. Instead, structural cues are learned from a category-specific dataset to complete partial point clouds of objects. We demonstrate the effectiveness of our approach on both synthetic ShapeNet data and real-world ScanNet data, showing that P2C produces comparable results to methods trained with complete shapes, and outperforms methods learned with multiple partial observations. Code is available at https://github.com/CuiRuikai/Partial2Complete. △ Less

Submitted 27 July, 2023; originally announced July 2023.

Comments: Accepted to ICCV 2023

Dual-Functional MIMO Beamforming Optimization for RIS-Aided Integrated Sensing and Communication

Authors: Xin Zhao, Heng Liu, Shiqi Gong, Xin Ju, Chengwen Xing, Nan Zhao

Abstract: Aiming at providing wireless communication systems with environment-perceptive capacity, emerging integrated sensing and communication (ISAC) technologies face multiple difficulties, especially in balancing the performance trade-off between the communication and radar functions. In this paper, we introduce a reconfigurable intelligent surface (RIS) to assist both data transmission and target detec… ▽ More Aiming at providing wireless communication systems with environment-perceptive capacity, emerging integrated sensing and communication (ISAC) technologies face multiple difficulties, especially in balancing the performance trade-off between the communication and radar functions. In this paper, we introduce a reconfigurable intelligent surface (RIS) to assist both data transmission and target detection in a dual-functional ISAC system. To formulate a general optimization framework, diverse communication performance metrics have been taken into account including famous capacity maximization and mean-squared error (MSE) minimization. Whereas the target detection process is modeled as a general likelihood ratio test (GLRT) due to the practical limitations, and the monotonicity of the corresponding detection probability is proved. For the single-user and single-target (SUST) scenario, the minimum transmit power of the ISAC transceiver has been revealed. By exploiting the optimal conditions of the BS design, we validate that the BS is able to realize the maximum power allocation scheme and derive the optimal BS precoder in a semi-closed form. Moreover, an alternating direction method of multipliers (ADMM) based RIS design is proposed to address the optimization of unit-modulus RIS phase shifts. For the sake of further enhancing computational efficiency, we also develop a low-complexity RIS design based on Riemannian gradient descent. Furthermore, the ISAC transceiver design for the multiple-users and multiple-targets (MUMT) scenario is also investigated, where a zero-forcing (ZF) radar receiver is adopted to cancel the interferences. Then optimal BS precoder is derived under the maximum power allocation scheme, and the RIS phase shifts can be optimized by extending the proposed ADMM-based RIS design. Numerical simulation results verify the performance of our proposed transceiver designs. △ Less

Submitted 17 July, 2023; originally announced July 2023.

Comments: 30 pages, 8 figures, manuscript submitted to IEEE TCOM

OpenSiteRec: An Open Dataset for Site Recommendation

Authors: Xinhang Li, Xiangyu Zhao, Ye**g Wang, Yu Liu, Yong Li, Cheng Long, Yong Zhang, Chunxiao Xing

Abstract: As a representative information retrieval task, site recommendation, which aims at predicting the optimal sites for a brand or an institution to open new branches in an automatic data-driven way, is beneficial and crucial for brand development in modern business. However, there is no publicly available dataset so far and most existing approaches are limited to an extremely small scope of brands, w… ▽ More As a representative information retrieval task, site recommendation, which aims at predicting the optimal sites for a brand or an institution to open new branches in an automatic data-driven way, is beneficial and crucial for brand development in modern business. However, there is no publicly available dataset so far and most existing approaches are limited to an extremely small scope of brands, which seriously hinders the research on site recommendation. Therefore, we collect, construct and release an open comprehensive dataset, namely OpenSiteRec, to facilitate and promote the research on site recommendation. Specifically, OpenSiteRec leverages a heterogeneous graph schema to represent various types of real-world entities and relations in four international metropolises. To evaluate the performance of the existing general methods on the site recommendation task, we conduct benchmarking experiments of several representative recommendation models on OpenSiteRec. Furthermore, we also highlight the potential application directions to demonstrate the wide applicability of OpenSiteRec. We believe that our OpenSiteRec dataset is significant and anticipated to encourage the development of advanced methods for site recommendation. OpenSiteRec is available online at https://OpenSiteRec.github.io/. △ Less

Submitted 3 July, 2023; originally announced July 2023.

Codes with Biochemical Constraints and Single Error Correction for DNA-Based Data Storage

Authors: Shu Liu, Yaqian Zhang

Abstract: In DNA-based data storage, DNA codes with biochemical constraints and error correction are designed to protect data reliability. Single-stranded DNA sequences with secondary structure avoidance (SSA) help to avoid undesirable secondary structures which may cause chemical inactivity. Homopolymer run-length limit and GC-balanced limit also help to reduce the error probability of DNA sequences during… ▽ More In DNA-based data storage, DNA codes with biochemical constraints and error correction are designed to protect data reliability. Single-stranded DNA sequences with secondary structure avoidance (SSA) help to avoid undesirable secondary structures which may cause chemical inactivity. Homopolymer run-length limit and GC-balanced limit also help to reduce the error probability of DNA sequences during synthesizing and sequencing. In this letter, based on a recent work \cite{bib7}, we construct DNA codes free of secondary structures of stem length $\geq m$ and have homopolymer run-length $\leq\ell$ for odd $m\leq11$ and $\ell\geq3$ with rate $1+\log_2ρ_m-3/(2^{\ell-1}+\ell+1)$, where $ρ_m$ is in Table \ref{tm}. In particular, when $m=3$, $\ell=4$, its rate tends to 1.3206 bits/nt, beating a previous work by Benerjee {\it et al.}. We also construct DNA codes with all of the above three constraints as well as single error correction. At last, codes with GC-locally balanced constraint are presented. △ Less

Submitted 1 July, 2023; originally announced July 2023.

Multimodal Audio-textual Architecture for Robust Spoken Language Understanding

Authors: Anderson R. Avila, Mehdi Rezagholizadeh, Chao Xing

Abstract: Recent voice assistants are usually based on the cascade spoken language understanding (SLU) solution, which consists of an automatic speech recognition (ASR) engine and a natural language understanding (NLU) system. Because such approach relies on the ASR output, it often suffers from the so-called ASR error propagation. In this work, we investigate impacts of this ASR error propagation on state-… ▽ More Recent voice assistants are usually based on the cascade spoken language understanding (SLU) solution, which consists of an automatic speech recognition (ASR) engine and a natural language understanding (NLU) system. Because such approach relies on the ASR output, it often suffers from the so-called ASR error propagation. In this work, we investigate impacts of this ASR error propagation on state-of-the-art NLU systems based on pre-trained language models (PLM), such as BERT and RoBERTa. Moreover, a multimodal language understanding (MLU) module is proposed to mitigate SLU performance degradation caused by errors present in the ASR transcript. The MLU benefits from self-supervised features learned from both audio and text modalities, specifically Wav2Vec for speech and Bert/RoBERTa for language. Our MLU combines an encoder network to embed the audio signal and a text encoder to process text transcripts followed by a late fusion layer to fuse audio and text logits. We found that the proposed MLU showed to be robust towards poor quality ASR transcripts, while the performance of BERT and RoBERTa are severely compromised. Our model is evaluated on five tasks from three SLU datasets and robustness is tested using ASR transcripts from three ASR engines. Results show that the proposed approach effectively mitigates the ASR error propagation problem, surpassing the PLM models' performance across all datasets for the academic ASR engine. △ Less

Submitted 13 June, 2023; v1 submitted 11 June, 2023; originally announced June 2023.

Explicit Construction of q-ary 2-deletion Correcting Codes with Low Redundancy

Authors: Shu Liu, Ivan Tjuawinata, Abstract: We consider the problem of efficient construction of q-ary 2-deletion correcting codes with low redundancy. We show that our construction requires less redundancy than any existing efficiently encodable q-ary 2-deletion correcting codes. Precisely speaking, we present an explicit construction of a q-ary 2-deletion correcting code with redundancy 5 log(n)+10log(log(n)) + 3 log(q)+O(1). Using a mino… ▽ More We consider the problem of efficient construction of q-ary 2-deletion correcting codes with low redundancy. We show that our construction requires less redundancy than any existing efficiently encodable q-ary 2-deletion correcting codes. Precisely speaking, we present an explicit construction of a q-ary 2-deletion correcting code with redundancy 5 log(n)+10log(log(n)) + 3 log(q)+O(1). Using a minor modification to the original construction, we obtain an efficiently encodable q-ary 2-deletion code that is efficiently list-decodable. Similarly, we show that our construction of list-decodable code requires a smaller redundancy compared to any existing list-decodable codes. To obtain our sketches, we transform a q-ary codeword to a binary string which can then be used as an input to the underlying base binary sketch. This is then complemented with additional q-ary sketches that the original q-ary codeword is required to satisfy. In other words, we build our codes via a binary 2-deletion code as a black-box. Finally we utilize the binary 2-deletion code proposed by Guruswami and Hastad to our construction to obtain the main result of this paper. △ Less

Submitted 9 June, 2023; v1 submitted 5 June, 2023; originally announced June 2023.

MuseCoco: Generating Symbolic Music from Text

Authors: Peiling Lu, Xin Xu, Chenfei Kang, Botao Yu, Chengyi Xing, Xu Tan, Jiang Bian

Abstract: Generating music from text descriptions is a user-friendly mode since the text is a relatively easy interface for user engagement. While some approaches utilize texts to control music audio generation, editing musical elements in generated audio is challenging for users. In contrast, symbolic music offers ease of editing, making it more accessible for users to manipulate specific musical elements.… ▽ More Generating music from text descriptions is a user-friendly mode since the text is a relatively easy interface for user engagement. While some approaches utilize texts to control music audio generation, editing musical elements in generated audio is challenging for users. In contrast, symbolic music offers ease of editing, making it more accessible for users to manipulate specific musical elements. In this paper, we propose MuseCoco, which generates symbolic music from text descriptions with musical attributes as the bridge to break down the task into text-to-attribute understanding and attribute-to-music generation stages. MuseCoCo stands for Music Composition Copilot that empowers musicians to generate music directly from given text descriptions, offering a significant improvement in efficiency compared to creating music entirely from scratch. The system has two main advantages: Firstly, it is data efficient. In the attribute-to-music generation stage, the attributes can be directly extracted from music sequences, making the model training self-supervised. In the text-to-attribute understanding stage, the text is synthesized and refined by ChatGPT based on the defined attribute templates. Secondly, the system can achieve precise control with specific attributes in text descriptions and offers multiple control options through attribute-conditioned or text-conditioned approaches. MuseCoco outperforms baseline systems in terms of musicality, controllability, and overall score by at least 1.27, 1.08, and 1.32 respectively. Besides, there is a notable enhancement of about 20% in objective control accuracy. In addition, we have developed a robust large-scale model with 1.2 billion parameters, showcasing exceptional controllability and musicality. △ Less

Submitted 31 May, 2023; originally announced June 2023.

Constructions of $k$-uniform states in heterogeneous systems

Authors: Keqin Feng, Lingfei **, Chen Yuan

Abstract: A pure quantum state of $n$ parties associated with the Hilbert space $\CC^{d_1}\otimes \CC^{d_2}\otimes\cdots\otimes \CC^{d_n}$ is called $k$-uniform if all the reductions to $k$-parties are maximally mixed. The $n$ partite system is called homogenous if the local dimension $d_1=d_2=\cdots=d_n$, while it is called heterogeneous if the local dimension are not all equal. $k$-uniform sates play an i… ▽ More A pure quantum state of $n$ parties associated with the Hilbert space $\CC^{d_1}\otimes \CC^{d_2}\otimes\cdots\otimes \CC^{d_n}$ is called $k$-uniform if all the reductions to $k$-parties are maximally mixed. The $n$ partite system is called homogenous if the local dimension $d_1=d_2=\cdots=d_n$, while it is called heterogeneous if the local dimension are not all equal. $k$-uniform sates play an important role in quantum information theory. There are many progress in characterizing and constructing $k$-uniform states in homogeneous systems. However, the study of entanglement for heterogeneous systems is much more challenging than that for the homogeneous case. There are very few results known for the $k$-uniform states in heterogeneous systems for $k>3$. We present two general methods to construct $k$-uniform states in the heterogeneous systems for general $k$. The first construction is derived from the error correcting codes by establishing a connection between irredundant mixed orthogonal arrays and error correcting codes. We can produce many new $k$-uniform states such that the local dimension of each subsystem can be a prime power. The second construction is derived from a matrix $H$ meeting the condition that $H_{A\times \bar{A}}+H^T_{\bar{A}\times A}$ has full rank for any row index set $A$ of size $k$. These matrix construction can provide more flexible choices for the local dimensions, i.e., the local dimensions can be any integer (not necessarily prime power) subject to some constraints. Our constructions imply that for any positive integer $k$, one can construct $k$-uniform states of a heterogeneous system in many different Hilbert spaces. △ Less

Submitted 22 May, 2023; originally announced May 2023.

Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotations

Authors: Vibashan VS, Ning Yu, Chen Xing, Can Qin, Mingfei Gao, Juan Carlos Niebles, Vishal M. Patel, Ran Xu

Abstract: Existing instance segmentation models learn task-specific information using manual mask annotations from base (training) categories. These mask annotations require tremendous human effort, limiting the scalability to annotate novel (new) categories. To alleviate this problem, Open-Vocabulary (OV) methods leverage large-scale image-caption pairs and vision-language models to learn novel categories.… ▽ More Existing instance segmentation models learn task-specific information using manual mask annotations from base (training) categories. These mask annotations require tremendous human effort, limiting the scalability to annotate novel (new) categories. To alleviate this problem, Open-Vocabulary (OV) methods leverage large-scale image-caption pairs and vision-language models to learn novel categories. In summary, an OV method learns task-specific information using strong supervision from base annotations and novel category information using weak supervision from image-captions pairs. This difference between strong and weak supervision leads to overfitting on base categories, resulting in poor generalization towards novel categories. In this work, we overcome this issue by learning both base and novel categories from pseudo-mask annotations generated by the vision-language model in a weakly supervised manner using our proposed Mask-free OVIS pipeline. Our method automatically generates pseudo-mask annotations by leveraging the localization ability of a pre-trained vision-language model for objects present in image-caption pairs. The generated pseudo-mask annotations are then used to supervise an instance segmentation model, freeing the entire pipeline from any labour-expensive instance-level annotations and overfitting. Our extensive experiments show that our method trained with just pseudo-masks significantly improves the mAP scores on the MS-COCO dataset and OpenImages dataset compared to the recent state-of-the-art methods trained with manual masks. Codes and models are provided in https://vibashan.github.io/ovis-web/. △ Less

Submitted 29 March, 2023; originally announced March 2023.

Comments: Accepted to CVPR 2023. Project site: https://vibashan.github.io/ovis-web/

IMF: Interactive Multimodal Fusion Model for Link Prediction

Authors: Xinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang, Chunxiao Xing

Abstract: Link prediction aims to identify potential missing triples in knowledge graphs. To get better results, some recent studies have introduced multimodal information to link prediction. However, these methods utilize multimodal information separately and neglect the complicated interaction between different modalities. In this paper, we aim at better modeling the inter-modality information and thus in… ▽ More Link prediction aims to identify potential missing triples in knowledge graphs. To get better results, some recent studies have introduced multimodal information to link prediction. However, these methods utilize multimodal information separately and neglect the complicated interaction between different modalities. In this paper, we aim at better modeling the inter-modality information and thus introduce a novel Interactive Multimodal Fusion (IMF) model to integrate knowledge from different modalities. To this end, we propose a two-stage multimodal fusion framework to preserve modality-specific knowledge as well as take advantage of the complementarity between different modalities. Instead of directly projecting different modalities into a unified space, our multimodal fusion module limits the representations of different modalities independent while leverages bilinear pooling for fusion and incorporates contrastive learning as additional constraints. Furthermore, the decision fusion module delivers the learned weighted average over the predictions of all modalities to better incorporate the complementarity of different modalities. Our approach has been demonstrated to be effective through empirical evaluations on several real-world datasets. The implementation code is available online at https://github.com/HestiaSky/IMF-Pytorch. △ Less

Submitted 19 March, 2023; originally announced March 2023.

Comments: 9 pages, 7 figures, 4 tables, WWW'2023

GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation

Authors: Can Qin, Ning Yu, Chen Xing, Shu Zhang, Zeyuan Chen, Stefano Ermon, Yun Fu, Caiming Xiong, Ran Xu

Abstract: Text-to-image (T2I) models based on diffusion processes have achieved remarkable success in controllable image generation using user-provided captions. However, the tight coupling between the current text encoder and image decoder in T2I models makes it challenging to replace or upgrade. Such changes often require massive fine-tuning or even training from scratch with the prohibitive expense. To a… ▽ More Text-to-image (T2I) models based on diffusion processes have achieved remarkable success in controllable image generation using user-provided captions. However, the tight coupling between the current text encoder and image decoder in T2I models makes it challenging to replace or upgrade. Such changes often require massive fine-tuning or even training from scratch with the prohibitive expense. To address this problem, we propose GlueGen, which applies a newly proposed GlueNet model to align features from single-modal or multi-modal encoders with the latent space of an existing T2I model. The approach introduces a new training objective that leverages parallel corpora to align the representation spaces of different encoders. Empirical results show that GlueNet can be trained efficiently and enables various capabilities beyond previous state-of-the-art models: 1) multilingual language models such as XLM-Roberta can be aligned with existing T2I models, allowing for the generation of high-quality images from captions beyond English; 2) GlueNet can align multi-modal encoders such as AudioCLIP with the Stable Diffusion model, enabling sound-to-image generation; 3) it can also upgrade the current text encoder of the latent diffusion model for challenging case generation. By the alignment of various feature representations, the GlueNet allows for flexible and efficient integration of new functionality into existing T2I models and sheds light on X-to-image (X2I) generation. △ Less

Submitted 2 November, 2023; v1 submitted 17 March, 2023; originally announced March 2023.

Comments: ICCV 2023

A novel failure indexing approach with run-time values of program variables

Authors: Yi Song, Xihao Zhang, Xiaoyuan Xie, Quanming Liu, Ruizhi Gao, Chenliang Xing

Abstract: Failures with different root causes can disturb multi-fault localization significantly, therefore, dividing failures into distinct groups according to the responsible faults is highly important. In such a failure indexing task, the crux lies in the failure proximity, which involves two points, i.e., how to effectively represent failures (e.g., extract the signature of failures) and how to properly… ▽ More Failures with different root causes can disturb multi-fault localization significantly, therefore, dividing failures into distinct groups according to the responsible faults is highly important. In such a failure indexing task, the crux lies in the failure proximity, which involves two points, i.e., how to effectively represent failures (e.g., extract the signature of failures) and how to properly measure the distance between the proxies for those failures. Existing studies have proposed a variety of failure proximities. The prevalent of them extract signatures of failures from execution coverage or suspiciousness ranking lists, and accordingly employ the Euclid or the Kendall tau distances. However, such strategies may not properly reflect the essential characteristics of failures, thus resulting in unsatisfactory effectiveness. In this paper, we propose a new failure proximity, namely, program variable-based failure proximity, and based on which present a novel failure indexing approach. Specifically, the proposed approach utilizes the run-time values of program variables to represent failures, and designs a set of rules to measure the similarity between them. Experimental results demonstrate the competitiveness of the proposed approach: it can achieve 44.12% and 27.59% improvements in faults number estimation, as well as 47.30% and 26.93% improvements in clustering effectiveness, compared with the state-of-the-art technique in this field, in simulated and real-world environments, respectively. △ Less

Submitted 2 February, 2023; originally announced February 2023.

ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding

Authors: Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, Silvio Savarese

Abstract: The recognition capabilities of current state-of-the-art 3D models are limited by datasets with a small number of annotated data and a pre-defined set of categories. In its 2D counterpart, recent advances have shown that similar problems can be significantly alleviated by employing knowledge from other modalities, such as language. Inspired by this, leveraging multimodal information for 3D modalit… ▽ More The recognition capabilities of current state-of-the-art 3D models are limited by datasets with a small number of annotated data and a pre-defined set of categories. In its 2D counterpart, recent advances have shown that similar problems can be significantly alleviated by employing knowledge from other modalities, such as language. Inspired by this, leveraging multimodal information for 3D modality could be promising to improve 3D understanding under the restricted data regime, but this line of research is not well studied. Therefore, we introduce ULIP to learn a unified representation of images, texts, and 3D point clouds by pre-training with object triplets from the three modalities. To overcome the shortage of training triplets, ULIP leverages a pre-trained vision-language model that has already learned a common visual and textual space by training with massive image-text pairs. Then, ULIP learns a 3D representation space aligned with the common image-text space, using a small number of automatically synthesized triplets. ULIP is agnostic to 3D backbone networks and can easily be integrated into any 3D architecture. Experiments show that ULIP effectively improves the performance of multiple recent 3D backbones by simply pre-training them on ShapeNet55 using our framework, achieving state-of-the-art performance in both standard 3D classification and zero-shot 3D classification on ModelNet40 and ScanObjectNN. ULIP also improves the performance of PointMLP by around 3% in 3D classification on ScanObjectNN, and outperforms PointCLIP by 28.8% on top-1 accuracy for zero-shot 3D classification on ModelNet40. Our code and pre-trained models are released at https://github.com/salesforce/ULIP. △ Less

Submitted 12 June, 2023; v1 submitted 9 December, 2022; originally announced December 2022.

Comments: Accepted by CVPR 2023

A Lower Bound on the List-Decodability of Insdel Codes

Authors: Shu Liu, Ivan Tjuawinata, Abstract: For codes equipped with metrics such as Hamming metric, symbol pair metric or cover metric, the Johnson bound guarantees list-decodability of such codes. That is, the Johnson bound provides a lower bound on the list-decoding radius of a code in terms of its relative minimum distance $δ$, list size $L$ and the alphabet size $q.$ For study of list-decodability of codes with insertion and deletion er… ▽ More For codes equipped with metrics such as Hamming metric, symbol pair metric or cover metric, the Johnson bound guarantees list-decodability of such codes. That is, the Johnson bound provides a lower bound on the list-decoding radius of a code in terms of its relative minimum distance $δ$, list size $L$ and the alphabet size $q.$ For study of list-decodability of codes with insertion and deletion errors (we call such codes insdel codes), it is natural to ask the open problem whether there is also a Johnson-type bound. The problem was first investigated by Wachter-Zeh and the result was amended by Hayashi and Yasunaga where a lower bound on the list-decodability for insdel codes was derived. The main purpose of this paper is to move a step further towards solving the above open problem. In this work, we provide a new lower bound for the list-decodability of an insdel code. As a consequence, we show that unlike the Johnson bound for codes under other metrics that is tight, the bound on list-decodability of insdel codes given by Hayashi and Yasunaga is not tight. Our main idea is to show that if an insdel code with a given Levenshtein distance $d$ is not list-decodable with list size $L$, then the list decoding radius is lower bounded by a bound involving $L$ and $d$. In other words, if the list decoding radius is less than this lower bound, the code must be list-decodable with list size $L$. At the end of the paper we use such bound to provide an insdel-list-decodability bound for various well-known codes, which has not been extensively studied before. △ Less

Submitted 12 August, 2023; v1 submitted 12 November, 2022; originally announced November 2022.

Binary sequences with a low correlation via cyclotomic function fields with odd characteristics

Authors: Lingfei **, Liming Ma, Abstract: Sequences with a low correlation have very important applications in communications, cryptography, and compressed sensing. In the literature, many efforts have been made to construct good sequences with various lengths where binary sequences attracts great attention. As a result, various constructions of good binary sequences have been proposed. However, most of the known constructions made use of… ▽ More Sequences with a low correlation have very important applications in communications, cryptography, and compressed sensing. In the literature, many efforts have been made to construct good sequences with various lengths where binary sequences attracts great attention. As a result, various constructions of good binary sequences have been proposed. However, most of the known constructions made use of the multiplicative cyclic group structure of finite field $\mathbb{F}_{p^n}$ for a prime $p$ and a positive integer $n$. In fact, all $p^n+1$ rational places including the place at infinity of the rational function field over $\mathbb{F}_{p^n}$ form a cyclic structure under an automorphism of order $p^n+1$. In this paper, we make use of this cyclic structure to provide an explicit construction of binary sequences with a low correlation of length $p^n+1$ via cyclotomic function fields over $\mathbb{F}_{p^n}$ for any odd prime $p$. Each family of binary sequences has size $p^n-2$ and its correlation is upper bounded by $4+\lfloor 2\cdot p^{n/2}\rfloor$. To the best of our knowledge, this is the first construction of binary sequences with a low correlation of length $p^n+1$ for odd prime $p$. Moreover, our sequences can be constructed explicitly and have competitive parameters. △ Less

Submitted 23 October, 2022; originally announced October 2022.

Comments: arXiv admin note: text overlap with arXiv:2107.11766

Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning

Authors: Xiangyu Peng, Chen Xing, Prafulla Kumar Choubey, Chien-Sheng Wu, Caiming Xiong

Abstract: Prompt tuning approaches, which learn task-specific soft prompts for a downstream task conditioning on frozen pre-trained models, have attracted growing interest due to its parameter efficiency. With large language models and sufficient training data, prompt tuning performs comparably to full-model tuning. However, with limited training samples in few-shot settings, prompt tuning fails to match th… ▽ More Prompt tuning approaches, which learn task-specific soft prompts for a downstream task conditioning on frozen pre-trained models, have attracted growing interest due to its parameter efficiency. With large language models and sufficient training data, prompt tuning performs comparably to full-model tuning. However, with limited training samples in few-shot settings, prompt tuning fails to match the performance of full-model fine-tuning. In this work, we focus on improving the few-shot performance of prompt tuning by transferring knowledge from soft prompts of source tasks. Recognizing the good generalization capabilities of ensemble methods in low-data regime, we first experiment and show that a simple ensemble of model predictions based on different source prompts, outperforms existing multi-prompt knowledge transfer approaches such as source prompt fusion in the few-shot setting. Motivated by this observation, we further investigate model ensembles and propose Sample-specific Ensemble of Source Models (SESoM). SESoM learns to adjust the contribution of each source model for each target sample separately when ensembling source model outputs. Through this way, SESoM inherits the superior generalization of model ensemble approaches and simultaneously captures the sample-specific competence of each source prompt. We conduct experiments across a diverse set of eight NLP tasks using models of different scales (T5-{base, large, XL}) and find that SESoM consistently outperforms the existing models of the same as well as larger parametric scale by a large margin. △ Less

Submitted 1 March, 2023; v1 submitted 22 October, 2022; originally announced October 2022.

A Unified Joint Optimization of Training Sequences and Transceivers Based on Matrix-Monotonic Optimization

Authors: Chengwen Xing, Tao Yu, **peng Song, Zhong Zheng, Lian Zhao, Lajos Hanzo

Abstract: Channel estimation and data transmission constitute the most fundamental functional modules of multiple-input multiple-output (MIMO) communication systems. The underlying key tasks corresponding to these modules are training sequence optimization and transceiver optimization. Hence, we jointly optimize the linear transmit precoder and the training sequence of MIMO systems using the metrics of thei… ▽ More Channel estimation and data transmission constitute the most fundamental functional modules of multiple-input multiple-output (MIMO) communication systems. The underlying key tasks corresponding to these modules are training sequence optimization and transceiver optimization. Hence, we jointly optimize the linear transmit precoder and the training sequence of MIMO systems using the metrics of their effective mutual information (MI), effective mean squared error (MSE), effective weighted MI, effective weighted MSE, as well as their effective generic Schur-convex and Schur-concave functions. Both statistical channel state information (CSI) and estimated CSI are considered at the transmitter in the joint optimization. A unified framework termed as joint matrix-monotonic optimization is proposed. Based on this, the optimal precoder matrix and training matrix structures can be derived for both CSI scenarios. Then, based on the optimal matrix structures, our linear transceivers and their training sequences can be jointly optimized. Compared to state-of-the-art benchmark algorithms, the proposed algorithms visualize the bold explicit relationships between the attainable system performance of our linear transceivers conceived and their training sequences, leading to implementation ready recipes. Finally, several numerical results are provided, which corroborate our theoretical results and demonstrate the compelling benefits of our proposed pilot-aided MIMO solutions. △ Less

Submitted 17 July, 2023; v1 submitted 22 September, 2022; originally announced September 2022.

Comments: 39 pages, 9 figures, 1 table, manuscript accepted in IEEE TVT

A new construction of nonlinear codes via algebraic function fields

Authors: Shu Liu, Liming Ma, Ting-Yi Wu, Abstract: In coding theory, constructing codes with good parameters is one of the most important and fundamental problems. Though a great many of good codes have been produced, most of them are defined over alphabets of sizes equal to prime powers. In this paper, we provide a new explicit construction of $(q+1)$-ary nonlinear codes via algebraic function fields, where $q$ is a prime power. Our codes are con… ▽ More In coding theory, constructing codes with good parameters is one of the most important and fundamental problems. Though a great many of good codes have been produced, most of them are defined over alphabets of sizes equal to prime powers. In this paper, we provide a new explicit construction of $(q+1)$-ary nonlinear codes via algebraic function fields, where $q$ is a prime power. Our codes are constructed by evaluations of rational functions at all rational places of the algebraic function field. Compared with algebraic geometry codes, the main difference is that we allow rational functions to be evaluated at pole places. After evaluating rational functions from a union of Riemann-Roch spaces, we obtain a family of nonlinear codes over the alphabet $\mathbb{F}_{q}\cup \{\infty\}$. It turns out that our codes have better parameters than those obtained from MDS codes or good algebraic geometry codes via code alphabet extension and restriction. △ Less

Submitted 30 August, 2022; originally announced August 2022.

DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization

Authors: Xinlin Li, Bang Liu, Rui Heng Yang, Vanessa Courville, Chao Xing, Vahid Partovi Nia

Abstract: Efficiently deploying deep neural networks on low-resource edge devices is challenging due to their ever-increasing resource requirements. To address this issue, researchers have proposed multiplication-free neural networks, such as Power-of-Two quantization, or also known as Shift networks, which aim to reduce memory usage and simplify computation. However, existing low-bit Shift networks are not… ▽ More Efficiently deploying deep neural networks on low-resource edge devices is challenging due to their ever-increasing resource requirements. To address this issue, researchers have proposed multiplication-free neural networks, such as Power-of-Two quantization, or also known as Shift networks, which aim to reduce memory usage and simplify computation. However, existing low-bit Shift networks are not as accurate as their full-precision counterparts, typically suffering from limited weight range encoding schemes and quantization loss. In this paper, we propose the DenseShift network, which significantly improves the accuracy of Shift networks, achieving competitive performance to full-precision networks for vision and speech applications. In addition, we introduce a method to deploy an efficient DenseShift network using non-quantized floating-point activations, while obtaining 1.6X speed-up over existing methods. To achieve this, we demonstrate that zero-weight values in low-bit Shift networks do not contribute to model capacity and negatively impact inference computation. To address this issue, we propose a zero-free shifting mechanism that simplifies inference and increases model capacity. We further propose a sign-scale decomposition design to enhance training efficiency and a low-variance random initialization strategy to improve the model's transfer learning performance. Our extensive experiments on various computer vision and speech tasks demonstrate that DenseShift outperforms existing low-bit multiplication-free networks and achieves competitive performance compared to full-precision networks. Furthermore, our proposed approach exhibits strong transfer learning performance without a drop in accuracy. Our code was released on GitHub. △ Less

Submitted 24 October, 2023; v1 submitted 20 August, 2022; originally announced August 2022.

Good locally repairable codes via propagation rules

Authors: Shu Liu, Liming Ma, Tingyi Wu, Abstract: In classical coding theory, it is common to construct new codes via propagation rules. There are various propagation rules to construct classical block codes. However, propagation rules have not been extensively explored for constructions of locally repairable codes. In this paper, we introduce a few propagation rules to construct good locally repairable codes. To our surprise, these simple propag… ▽ More In classical coding theory, it is common to construct new codes via propagation rules. There are various propagation rules to construct classical block codes. However, propagation rules have not been extensively explored for constructions of locally repairable codes. In this paper, we introduce a few propagation rules to construct good locally repairable codes. To our surprise, these simple propagation rules produce a few interesting results. Firstly, by concatenating a locally repairable code as an inner code with a classical block code as an outer code, we obtain quite a few dimension-optimal binary locally repairable codes. Secondly, from this concatenation, we explicitly build a family of locally repairable codes that exceeds the Zyablov-type bound. Thirdly, by a lengthening propagation rule that adds some rows and columns from a parity-check matrix of a given linear code, we are able to produce a family of dimension-optimal binary locally repairable codes from the extended Hamming codes, and to convert a classical maximum distance separable (MDS) code into a Singleton-optimal locally repairable code. Furthermore, via the lengthening propagation rule, we greatly simplify the construction of a family of locally repairable codes in \cite[Theorem 5]{MX20} that breaks the asymptotic Gilbert-Varshamov bound. In addition, we make use of three other propagation rules to produce more dimension-optimal binary locally repairable codes. Finally, one of phenomena that we observe in this paper is that some trivial propagation rules in classical block codes do not hold anymore for locally repairable codes. △ Less

Submitted 8 August, 2022; originally announced August 2022.

LEO Satellite-Enabled Grant-Free Random Access with MIMO-OTFS

Authors: Boxiao Shen, Yongpeng Wu, Wenjun Zhang, Geoffrey Ye Li, Jian** An, Chengwen Xing

Abstract: This paper investigates joint channel estimation and device activity detection in the LEO satellite-enabled grant-free random access systems with large differential delay and Doppler shift. In addition, the multiple-input multiple-output (MIMO) with orthogonal time-frequency space modulation (OTFS) is utilized to combat the dynamics of the terrestrial-satellite link. To simplify the computation pr… ▽ More This paper investigates joint channel estimation and device activity detection in the LEO satellite-enabled grant-free random access systems with large differential delay and Doppler shift. In addition, the multiple-input multiple-output (MIMO) with orthogonal time-frequency space modulation (OTFS) is utilized to combat the dynamics of the terrestrial-satellite link. To simplify the computation process, we estimate the channel tensor in parallel along the delay dimension. Then, the deep learning and expectation-maximization approach are integrated into the generalized approximate message passing with cross-correlation--based Gaussian prior to capture the channel sparsity in the delay-Doppler-angle domain and learn the hyperparameters. Finally, active devices are detected by computing energy of the estimated channel. Simulation results demonstrate that the proposed algorithms outperform conventional methods. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Comments: This paper has been accepted for presentation at the IEEE GLOBECOM 2022. arXiv admin note: text overlap with arXiv:2202.13058

Low-bit Shift Network for End-to-End Spoken Language Understanding

Authors: Anderson R. Avila, Khalil Bibi, Rui Heng Yang, Xinlin Li, Chao Xing, Xiao Chen

Abstract: Deep neural networks (DNN) have achieved impressive success in multiple domains. Over the years, the accuracy of these models has increased with the proliferation of deeper and more complex architectures. Thus, state-of-the-art solutions are often computationally expensive, which makes them unfit to be deployed on edge computing platforms. In order to mitigate the high computation, memory, and pow… ▽ More Deep neural networks (DNN) have achieved impressive success in multiple domains. Over the years, the accuracy of these models has increased with the proliferation of deeper and more complex architectures. Thus, state-of-the-art solutions are often computationally expensive, which makes them unfit to be deployed on edge computing platforms. In order to mitigate the high computation, memory, and power requirements of inferring convolutional neural networks (CNNs), we propose the use of power-of-two quantization, which quantizes continuous parameters into low-bit power-of-two values. This reduces computational complexity by removing expensive multiplication operations and with the use of low-bit weights. ResNet is adopted as the building block of our solution and the proposed model is evaluated on a spoken language understanding (SLU) task. Experimental results show improved performance for shift neural network architectures, with our low-bit quantization achieving 98.76 \% on the test set which is comparable performance to its full-precision counterpart and state-of-the-art solutions. △ Less

Submitted 15 July, 2022; originally announced July 2022.

Comments: Accepted at INTERSPEECH 2022

Upper bounds on maximum lengths of Singleton-optimal locally repairable codes

Authors: Shu Liu, Tingyi Wu, Chen Yuan

Abstract: A locally repairable code is called Singleton-optimal if it achieves the Singleton-type bound. Such codes are of great theoretic interest in the study of locally repairable codes. In the recent years there has been a great amount of work on this topic. One of the main problems in this topic is to determine the largest length of a q-ary Singleton-optimal locally repairable code for given locality a… ▽ More A locally repairable code is called Singleton-optimal if it achieves the Singleton-type bound. Such codes are of great theoretic interest in the study of locally repairable codes. In the recent years there has been a great amount of work on this topic. One of the main problems in this topic is to determine the largest length of a q-ary Singleton-optimal locally repairable code for given locality and minimum distance. Unlike classical MDS codes, the maximum length of Singleton? Optimal locally repairable codes are very sensitive to minimum distance and locality. Thus, it is more challenging and complicated to investigate the maximum length of Singleton-optimal locally repairable codes. In literature, there has been already some research on this problem. However, most of work is concerned with some specific parameter regime such as small minimum distance and locality, and rely on the constraint that (r + 1)|n and recovery sets are disjoint, where r is locality and n is the code length. In this paper we study the problem for large range of parameters including the case where minimum distance is proportional to length. In addition, we also derive some upper bounds on the maximum length of Singleton-optimal locally repairable codes with small minimum distance by removing this constraint. It turns out that even without the constraint we still get better upper bounds for codes with small locality and distance compared with known results. Furthermore, based on our upper bounds for codes with small distance and locality and some propagation rule that we propose in this paper, we are able to derive some upper bounds for codes with relatively large distance and locality assuming that (r + 1)|n and recovery sets are disjoint. △ Less

Submitted 12 July, 2022; originally announced July 2022.

Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding

Authors: Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Duan Xinyu, Zhefeng Wang, Baoxing Huai, Xin Jiang, Qun Liu, Philippe Langlais

Abstract: There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language. This work concerns addressing two major problems in existing Arabic PLMs which constraint progress of the Arabic NLU and NLG fields.First, existing Arabic PLMs are not well-explored and their pre-trainig can be improved significantly using a more methodical approach. Second, there… ▽ More There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language. This work concerns addressing two major problems in existing Arabic PLMs which constraint progress of the Arabic NLU and NLG fields.First, existing Arabic PLMs are not well-explored and their pre-trainig can be improved significantly using a more methodical approach. Second, there is a lack of systematic and reproducible evaluation of these models in the literature. In this work, we revisit both the pre-training and evaluation of Arabic PLMs. In terms of pre-training, we explore improving Arabic LMs from three perspectives: quality of the pre-training data, size of the model, and incorporating character-level information. As a result, we release three new Arabic BERT-style models ( JABER, Char-JABER, and SABER), and two T5-style models (AT5S and AT5B). In terms of evaluation, we conduct a comprehensive empirical study to systematically evaluate the performance of existing state-of-the-art models on ALUE that is a leaderboard-powered benchmark for Arabic NLU tasks, and on a subset of the ARGEN benchmark for Arabic NLG tasks. We show that our models significantly outperform existing Arabic PLMs and achieve a new state-of-the-art performance on discriminative and generative Arabic NLU and NLG tasks. Our models and source code to reproduce of results will be made available shortly. △ Less

Submitted 21 May, 2022; originally announced May 2022.

GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learning

Authors: **yin Chen, Shulong Hu, Haibin Zheng, Changyou Xing, Guomin Zhang

Abstract: Penetration testing (PT) is an efficient network testing and vulnerability mining tool by simulating a hacker's attack for valuable information applied in some areas. Compared with manual PT, intelligent PT has become a dominating mainstream due to less time-consuming and lower labor costs. Unfortunately, RL-based PT is still challenged in real exploitation scenarios because the agent's action spa… ▽ More Penetration testing (PT) is an efficient network testing and vulnerability mining tool by simulating a hacker's attack for valuable information applied in some areas. Compared with manual PT, intelligent PT has become a dominating mainstream due to less time-consuming and lower labor costs. Unfortunately, RL-based PT is still challenged in real exploitation scenarios because the agent's action space is usually high-dimensional discrete, thus leading to algorithm convergence difficulty. Besides, most PT methods still rely on the decisions of security experts. Addressing the challenges, for the first time, we introduce expert knowledge to guide the agent to make better decisions in RL-based PT and propose a Generative Adversarial Imitation Learning-based generic intelligent Penetration testing framework, denoted as GAIL-PT, to solve the problems of higher labor costs due to the involvement of security experts and high-dimensional discrete action space. Specifically, first, we manually collect the state-action pairs to construct an expert knowledge base when the pre-trained RL / DRL model executes successful penetration testings. Second, we input the expert knowledge and the state-action pairs generated online by the different RL / DRL models into the discriminator of GAIL for training. At last, we apply the output reward of the discriminator to guide the agent to perform the action with a higher penetration success rate to improve PT's performance. Extensive experiments conducted on the real target host and simulated network scenarios show that GAIL-PT achieves the SOTA penetration performance against DeepExploit in exploiting actual target Metasploitable2 and Q-learning in optimizing penetration path, not only in small-scale with or without honey-pot network environments but also in the large-scale virtual network environment. △ Less

Submitted 5 April, 2022; originally announced April 2022.

Descriptellation: Deep Learned Constellation Descriptors

Authors: Chunwei Xing, Xinyu Sun, Andrei Cramariuc, Samuel Gull, Jen Jen Chung, Cesar Cadena, Roland Siegwart, Florian Tschopp

Abstract: Current descriptors for global localization often struggle under vast viewpoint or appearance changes. One possible improvement is the addition of topological information on semantic objects. However, handcrafted topological descriptors are hard to tune and not robust to environmental noise, drastic perspective changes, object occlusion or misdetections. To solve this problem, we formulate a learn… ▽ More Current descriptors for global localization often struggle under vast viewpoint or appearance changes. One possible improvement is the addition of topological information on semantic objects. However, handcrafted topological descriptors are hard to tune and not robust to environmental noise, drastic perspective changes, object occlusion or misdetections. To solve this problem, we formulate a learning-based approach by modelling semantically meaningful object constellations as graphs and using Deep Graph Convolution Networks to map a constellation to a descriptor. We demonstrate the effectiveness of our Deep Learned Constellation Descriptor (Descriptellation) on two real-world datasets. Although Descriptellation is trained on randomly generated simulation datasets, it shows good generalization abilities on real-world datasets. Descriptellation also outperforms state-of-the-art and handcrafted constellation descriptors for global localization, and is robust to different types of noise. The code is publicly available at https://github.com/ethz-asl/Descriptellation. △ Less

Submitted 14 September, 2022; v1 submitted 1 March, 2022; originally announced March 2022.

Random Access with Massive MIMO-OTFS in LEO Satellite Communications

Authors: Boxiao Shen, Yongpeng Wu, Jian** An, Chengwen Xing, Lian Zhao, Wenjun Zhang

Abstract: This paper considers the joint channel estimation and device activity detection in the grant-free random access systems, where a large number of Internet-of-Things devices intend to communicate with a low-earth orbit satellite in a sporadic way. In addition, the massive multiple-input multiple-output (MIMO) with orthogonal time-frequency space (OTFS) modulation is adopted to combat the dynamics of… ▽ More This paper considers the joint channel estimation and device activity detection in the grant-free random access systems, where a large number of Internet-of-Things devices intend to communicate with a low-earth orbit satellite in a sporadic way. In addition, the massive multiple-input multiple-output (MIMO) with orthogonal time-frequency space (OTFS) modulation is adopted to combat the dynamics of the terrestrial-satellite link. We first analyze the input-output relationship of the single-input single-output OTFS when the large delay and Doppler shift both exist, and then extend it to the grant-free random access with massive MIMO-OTFS. Next, by exploring the sparsity of channel in the delay-Doppler-angle domain, a two-dimensional pattern coupled hierarchical prior with the sparse Bayesian learning and covariance-free method (TDSBL-CF) is developed for the channel estimation. Then, the active devices are detected by computing the energy of the estimated channel. Finally, the generalized approximate message passing algorithm combined with the sparse Bayesian learning and two-dimensional convolution (ConvSBL-GAMP) is proposed to decrease the computations of the TDSBL-CF algorithm. Simulation results demonstrate that the proposed algorithms outperform conventional methods. △ Less

Submitted 9 August, 2022; v1 submitted 25 February, 2022; originally announced February 2022.

Comments: This paper has been accepted by IEEE JSAC Special Issue on Antenna Array Enabled Space/Air/Ground Communications and Networking

Massive Unsourced Random Access: Exploiting Angular Domain Sparsity

Authors: Xinyu Xie, Yongpeng Wu, Jian** An, Junyuan Gao, Wenjun Zhang, Chengwen Xing, Kai-Kit Wong, Chengshan Xiao

Abstract: This paper investigates the unsourced random access (URA) scheme to accommodate numerous machine-type users communicating to a base station equipped with multiple antennas. Existing works adopt a slotted transmission strategy to reduce system complexity; they operate under the framework of coupled compressed sensing (CCS) which concatenates an outer tree code to an inner compressed sensing code fo… ▽ More This paper investigates the unsourced random access (URA) scheme to accommodate numerous machine-type users communicating to a base station equipped with multiple antennas. Existing works adopt a slotted transmission strategy to reduce system complexity; they operate under the framework of coupled compressed sensing (CCS) which concatenates an outer tree code to an inner compressed sensing code for slot-wise message stitching. We suggest that by exploiting the MIMO channel information in the angular domain, redundancies required by the tree encoder/decoder in CCS can be removed to improve spectral efficiency, thereby an uncoupled transmission protocol is devised. To perform activity detection and channel estimation, we propose an expectation-maximization-aided generalized approximate message passing algorithm with a Markov random field support structure, which captures the inherent clustered sparsity structure of the angular domain channel. Then, message reconstruction in the form of a clustering decoder is performed by recognizing slot-distributed channels of each active user based on similarity. We put forward the slot-balanced K-means algorithm as the kernel of the clustering decoder, resolving constraints and collisions specific to the application scene. Extensive simulations reveal that the proposed scheme achieves a better error performance at high spectral efficiency compared to the CCS-based URA schemes. △ Less

Submitted 16 February, 2022; originally announced February 2022.

Comments: Accepted for publication in IEEE TCOM

Jointly Learning Knowledge Embedding and Neighborhood Consensus with Relational Knowledge Distillation for Entity Alignment

Authors: Xinhang Li, Yong Zhang, Chunxiao Xing

Abstract: Entity alignment aims at integrating heterogeneous knowledge from different knowledge graphs. Recent studies employ embedding-based methods by first learning the representation of Knowledge Graphs and then performing entity alignment via measuring the similarity between entity embeddings. However, they failed to make good use of the relation semantic information due to the trade-off problem caused… ▽ More Entity alignment aims at integrating heterogeneous knowledge from different knowledge graphs. Recent studies employ embedding-based methods by first learning the representation of Knowledge Graphs and then performing entity alignment via measuring the similarity between entity embeddings. However, they failed to make good use of the relation semantic information due to the trade-off problem caused by the different objectives of learning knowledge embedding and neighborhood consensus. To address this problem, we propose Relational Knowledge Distillation for Entity Alignment (RKDEA), a Graph Convolutional Network (GCN) based model equipped with knowledge distillation for entity alignment. We adopt GCN-based models to learn the representation of entities by considering the graph structure and incorporating the relation semantic information into GCN via knowledge distillation. Then, we introduce a novel adaptive mechanism to transfer relational knowledge so as to jointly learn entity embedding and neighborhood consensus. Experimental results on several benchmarking datasets demonstrate the effectiveness of our proposed model. △ Less

Submitted 24 January, 2022; originally announced January 2022.

Perlin Noise Improve Adversarial Robustness

Authors: Chengjun Tang, Kun Zhang, Chunfang Xing, Yong Ding, Zengmin Xu

Abstract: Adversarial examples are some special input that can perturb the output of a deep neural network, in order to make produce intentional errors in the learning algorithms in the production environment. Most of the present methods for generating adversarial examples require gradient information. Even universal perturbations that are not relevant to the generative model rely to some extent on gradient… ▽ More Adversarial examples are some special input that can perturb the output of a deep neural network, in order to make produce intentional errors in the learning algorithms in the production environment. Most of the present methods for generating adversarial examples require gradient information. Even universal perturbations that are not relevant to the generative model rely to some extent on gradient information. Procedural noise adversarial examples is a new way of adversarial example generation, which uses computer graphics noise to generate universal adversarial perturbations quickly while not relying on gradient information. Combined with the defensive idea of adversarial training, we use Perlin noise to train the neural network to obtain a model that can defend against procedural noise adversarial examples. In combination with the use of model fine-tuning methods based on pre-trained models, we obtain faster training as well as higher accuracy. Our study shows that procedural noise adversarial examples are defensible, but why procedural noise can generate adversarial examples and how to defend against other kinds of procedural noise adversarial examples that may emerge in the future remain to be investigated. △ Less

Submitted 26 December, 2021; originally announced December 2021.

Joint Device Detection, Channel Estimation, and Data Decoding with Collision Resolution for MIMO Massive Unsourced Random Access

Authors: Tianya Li, Yongpeng Wu, Mengfan Zheng, Wenjun Zhang, Chengwen Xing, Jian** An, Xiang-Gen Xia, Chengshan Xiao

Abstract: In this paper, we investigate a joint device activity detection (DAD), channel estimation (CE), and data decoding (DD) algorithm for multiple-input multiple-output (MIMO) massive unsourced random access (URA). Different from the state-of-the-art slotted transmission scheme, the data in the proposed framework is split into only two parts. A portion of the data is coded by compressed sensing (CS) an… ▽ More In this paper, we investigate a joint device activity detection (DAD), channel estimation (CE), and data decoding (DD) algorithm for multiple-input multiple-output (MIMO) massive unsourced random access (URA). Different from the state-of-the-art slotted transmission scheme, the data in the proposed framework is split into only two parts. A portion of the data is coded by compressed sensing (CS) and the rest is low-density-parity-check (LDPC) coded. In addition to being part of the data, information bits in the CS phase also undertake the task of interleaving pattern design and channel estimation (CE). The principle of interleave-division multiple access (IDMA) is exploited to reduce the interference among devices in the LDPC phase. Based on the belief propagation (BP) algorithm, a low-complexity iterative message passing (MP) algorithm is utilized to decode the data embedded in these two phases separately. Moreover, combined with successive interference cancellation (SIC), the proposed joint DAD-CE-DD algorithm is performed to further improve performance by utilizing the belief of each other. Additionally, based on the energy detection (ED) and sliding window protocol (SWP), we develop a collision resolution protocol to handle the codeword collision, a common issue in the URA system. In addition to the complexity reduction, the proposed algorithm exhibits a substantial performance enhancement compared to the state-of-the-art in terms of efficiency and accuracy. △ Less

Submitted 16 December, 2021; originally announced December 2021.

Comments: Accepted by IEEE JSAC special issue on Next Generation Multiple Access