-
SGCCNet: Single-Stage 3D Object Detector With Saliency-Guided Data Augmentation and Confidence Correction Mechanism
Authors:
Ao Liang,
Wenyu Chen,
Jian Fang,
Huaici Zhao
Abstract:
The single-stage point-based 3D object detectors have attracted widespread research interest due to their advantages of lightweight and fast inference speed. However, they still face challenges such as inadequate learning of low-quality objects (ILQ) and misalignment between localization accuracy and classification confidence (MLC). In this paper, we propose SGCCNet to alleviate these two issues.…
▽ More
The single-stage point-based 3D object detectors have attracted widespread research interest due to their advantages of lightweight and fast inference speed. However, they still face challenges such as inadequate learning of low-quality objects (ILQ) and misalignment between localization accuracy and classification confidence (MLC). In this paper, we propose SGCCNet to alleviate these two issues. For ILQ, SGCCNet adopts a Saliency-Guided Data Augmentation (SGDA) strategy to enhance the robustness of the model on low-quality objects by reducing its reliance on salient features. Specifically, We construct a classification task and then approximate the saliency scores of points by moving points towards the point cloud centroid in a differentiable process. During the training process, SGCCNet will be forced to learn from low saliency features through drop** points. Meanwhile, to avoid internal covariate shift and contextual features forgetting caused by drop** points, we add a geometric normalization module and skip connection block in each stage. For MLC, we design a Confidence Correction Mechanism (CCM) specifically for point-based multi-class detectors. This mechanism corrects the confidence of the current proposal by utilizing the predictions of other key points within the local region in the post-processing stage. Extensive experiments on the KITTI dataset demonstrate the generality and effectiveness of our SGCCNet. On the KITTI \textit{test} set, SGCCNet achieves $80.82\%$ for the metric of $AP_{3D}$ on the \textit{Moderate} level, outperforming all other point-based detectors, surpassing IA-SSD and Fast Point R-CNN by $2.35\%$ and $3.42\%$, respectively. Additionally, SGCCNet demonstrates excellent portability for other point-based detectors
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Blind Inversion using Latent Diffusion Priors
Authors:
Weimin Bai,
Siyi Chen,
Wenzheng Chen,
He Sun
Abstract:
Diffusion models have emerged as powerful tools for solving inverse problems due to their exceptional ability to model complex prior distributions. However, existing methods predominantly assume known forward operators (i.e., non-blind), limiting their applicability in practical settings where acquiring such operators is costly. Additionally, many current approaches rely on pixel-space diffusion m…
▽ More
Diffusion models have emerged as powerful tools for solving inverse problems due to their exceptional ability to model complex prior distributions. However, existing methods predominantly assume known forward operators (i.e., non-blind), limiting their applicability in practical settings where acquiring such operators is costly. Additionally, many current approaches rely on pixel-space diffusion models, leaving the potential of more powerful latent diffusion models (LDMs) underexplored. In this paper, we introduce LatentDEM, an innovative technique that addresses more challenging blind inverse problems using latent diffusion priors. At the core of our method is solving blind inverse problems within an iterative Expectation-Maximization (EM) framework: (1) the E-step recovers clean images from corrupted observations using LDM priors and a known forward model, and (2) the M-step estimates the forward operator based on the recovered images. Additionally, we propose two novel optimization techniques tailored for LDM priors and EM frameworks, yielding more accurate and efficient blind inversion results. As a general framework, LatentDEM supports both linear and non-linear inverse problems. Beyond common 2D image restoration tasks, it enables new capabilities in non-linear 3D inverse rendering problems. We validate LatentDEM's performance on representative 2D blind deblurring and 3D sparse-view reconstruction tasks, demonstrating its superior efficacy over prior arts.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations
Authors:
Weimin Bai,
Yifei Wang,
Wenzheng Chen,
He Sun
Abstract:
Diffusion models excel in solving imaging inverse problems due to their ability to model complex image priors. However, their reliance on large, clean datasets for training limits their practical use where clean data is scarce. In this paper, we propose EMDiffusion, an expectation-maximization (EM) approach to train diffusion models from corrupted observations. Our method alternates between recons…
▽ More
Diffusion models excel in solving imaging inverse problems due to their ability to model complex image priors. However, their reliance on large, clean datasets for training limits their practical use where clean data is scarce. In this paper, we propose EMDiffusion, an expectation-maximization (EM) approach to train diffusion models from corrupted observations. Our method alternates between reconstructing clean images from corrupted data using a known diffusion model (E-step) and refining diffusion model weights based on these reconstructions (M-step). This iterative process leads the learned diffusion model to gradually converge to the true clean data distribution. We validate our method through extensive experiments on diverse computational imaging tasks, including random inpainting, denoising, and deblurring, achieving new state-of-the-art performance.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Multi-Functional Beamforming Design for Integrated Sensing, Communication, and Computation
Authors:
Yapeng Zhao,
Qingqing Wu,
Wen Chen,
Yong Zeng,
Ruiqi Liu,
Weidong Mei,
Fen Hou,
Shaodan Ma
Abstract:
Integrated sensing and communication (ISAC) systems may face a heavy computation burden since the sensory data needs to be further processed. This paper studies a novel system that integrates sensing, communication, and computation, aiming to provide services for different objectives efficiently. This system consists of a multi-antenna multi-functional base station (BS), an edge server, a target,…
▽ More
Integrated sensing and communication (ISAC) systems may face a heavy computation burden since the sensory data needs to be further processed. This paper studies a novel system that integrates sensing, communication, and computation, aiming to provide services for different objectives efficiently. This system consists of a multi-antenna multi-functional base station (BS), an edge server, a target, and multiple singleantenna communication users. The BS needs to allocate the available resources to efficiently provide sensing, communication, and computation services. Due to the heavy service burden and limited power budget, the BS can partially offload the tasks to the nearby edge server instead of computing them locally. We consider the estimation of the target response matrix, a general problem in radar sensing, and utilize Cramer-Rao bound (CRB) as the corresponding performance metric. To tackle the non-convex optimization problem, we propose both semidefinite relaxation (SDR)-based alternating optimization and SDR-based successive convex approximation (SCA) algorithms to minimize the CRB of radar sensing while meeting the requirement of communication users and the need for task computing. Furthermore, we demonstrate that the optimal rankone solutions of both the alternating and SCA algorithms can be directly obtained via the solver or further constructed even when dealing with multiple functionalities. Simulation results show that the proposed algorithms can provide higher target estimation performance than state-of-the-art benchmarks while satisfying the communication and computation constraints.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
CURLS: Causal Rule Learning for Subgroups with Significant Treatment Effect
Authors:
Jiehui Zhou,
Linxiao Yang,
Xingyu Liu,
Xinyue Gu,
Liang Sun,
Wei Chen
Abstract:
In causal inference, estimating heterogeneous treatment effects (HTE) is critical for identifying how different subgroups respond to interventions, with broad applications in fields such as precision medicine and personalized advertising. Although HTE estimation methods aim to improve accuracy, how to provide explicit subgroup descriptions remains unclear, hindering data interpretation and strateg…
▽ More
In causal inference, estimating heterogeneous treatment effects (HTE) is critical for identifying how different subgroups respond to interventions, with broad applications in fields such as precision medicine and personalized advertising. Although HTE estimation methods aim to improve accuracy, how to provide explicit subgroup descriptions remains unclear, hindering data interpretation and strategic intervention management. In this paper, we propose CURLS, a novel rule learning method leveraging HTE, which can effectively describe subgroups with significant treatment effects. Specifically, we frame causal rule learning as a discrete optimization problem, finely balancing treatment effect with variance and considering the rule interpretability. We design an iterative procedure based on the minorize-maximization algorithm and solve a submodular lower bound as an approximation for the original. Quantitative experiments and qualitative case studies verify that compared with state-of-the-art methods, CURLS can find subgroups where the estimated and true effects are 16.1% and 13.8% higher and the variance is 12.0% smaller, while maintaining similar or better estimation accuracy and rule interpretability. Code is available at https://osf.io/zwp2k/.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Towards Robust Speech Representation Learning for Thousands of Languages
Authors:
William Chen,
Wangyou Zhang,
Yifan Peng,
Xinjian Li,
**chuan Tian,
Jiatong Shi,
Xuankai Chang,
Soumi Maiti,
Karen Livescu,
Shinji Watanabe
Abstract:
Self-supervised learning (SSL) has helped extend speech technologies to more languages by reducing the need for labeled data. However, models are still far from supporting the world's 7000+ languages. We propose XEUS, a Cross-lingual Encoder for Universal Speech, trained on over 1 million hours of data across 4057 languages, extending the language coverage of SSL models 4-fold. We combine 1 millio…
▽ More
Self-supervised learning (SSL) has helped extend speech technologies to more languages by reducing the need for labeled data. However, models are still far from supporting the world's 7000+ languages. We propose XEUS, a Cross-lingual Encoder for Universal Speech, trained on over 1 million hours of data across 4057 languages, extending the language coverage of SSL models 4-fold. We combine 1 million hours of speech from existing publicly accessible corpora with a newly created corpus of 7400+ hours from 4057 languages, which will be publicly released. To handle the diverse conditions of multilingual speech data, we augment the typical SSL masked prediction approach with a novel dereverberation objective, increasing robustness. We evaluate XEUS on several benchmarks, and show that it consistently outperforms or achieves comparable results to state-of-the-art (SOTA) SSL models across a variety of tasks. XEUS sets a new SOTA on the ML-SUPERB benchmark: it outperforms MMS 1B and w2v-BERT 2.0 v2 by 0.8% and 4.4% respectively, despite having less parameters or pre-training data. Checkpoints, code, and data are found in https://www.wavlab.org/activities/2024/xeus/.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
Physical Layer Deception with Non-Orthogonal Multiplexing
Authors:
Wenwen Chen,
Bin Han,
Yao Zhu,
Anke Schmeink,
Giuseppe Caire,
Hans D. Schotten
Abstract:
Physical layer security (PLS) is a promising technology to secure wireless communications by exploiting the physical properties of the wireless channel. However, the passive nature of PLS creates a significant imbalance between the effort required by eavesdroppers and legitimate users to secure data. To address this imbalance, in this article, we propose a novel framework of physical layer decepti…
▽ More
Physical layer security (PLS) is a promising technology to secure wireless communications by exploiting the physical properties of the wireless channel. However, the passive nature of PLS creates a significant imbalance between the effort required by eavesdroppers and legitimate users to secure data. To address this imbalance, in this article, we propose a novel framework of physical layer deception (PLD), which combines PLS with deception technologies to actively counteract wiretap** attempts. Combining a two-stage encoder with randomized ciphering and non-orthogonal multiplexing, the PLD approach enables the wireless communication system to proactively counter eavesdroppers with deceptive messages. Relying solely on the superiority of the legitimate channel over the eavesdrop** channel, the PLD framework can effectively protect the confidentiality of the transmitted messages, even against eavesdroppers who possess knowledge equivalent to that of the legitimate receiver. We prove the validity of the PLD framework with in-depth analyses and demonstrate its superiority over conventional PLS approaches with comprehensive numerical benchmarks.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
Emergent Crowd Grou** via Heuristic Self-Organization
Authors:
Xiao-Cheng Liao,
Wei-Neng Chen,
Xiang-Ling Chen,
Yi Mei
Abstract:
Modeling crowds has many important applications in games and computer animation. Inspired by the emergent following effect in real-life crowd scenarios, in this work, we develop a method for implicitly grou** moving agents. We achieve this by analyzing local information around each agent and rotating its preferred velocity accordingly. Each agent could automatically form an implicit group with i…
▽ More
Modeling crowds has many important applications in games and computer animation. Inspired by the emergent following effect in real-life crowd scenarios, in this work, we develop a method for implicitly grou** moving agents. We achieve this by analyzing local information around each agent and rotating its preferred velocity accordingly. Each agent could automatically form an implicit group with its neighboring agents that have similar directions. In contrast to an explicit group, there are no strict boundaries for an implicit group. If an agent's direction deviates from its group as a result of positional changes, it will autonomously exit the group or join another implicitly formed neighboring group. This implicit grou** is autonomously emergent among agents rather than deliberately controlled by the algorithm. The proposed method is compared with many crowd simulation models, and the experimental results indicate that our approach achieves the lowest congestion levels in some classic scenarios. In addition, we demonstrate that adjusting the preferred velocity of agents can actually reduce the dissimilarity between their actual velocity and the original preferred velocity. Our work is available online.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
Learning Granularity-Aware Affordances from Human-Object Interaction for Tool-Based Functional Gras** in Dexterous Robotics
Authors:
Fan Yang,
Wenrui Chen,
Kailun Yang,
Haoran Lin,
DongSheng Luo,
Conghui Tang,
Zhiyong Li,
Yaonan Wang
Abstract:
To enable robots to use tools, the initial step is teaching robots to employ dexterous gestures for touching specific areas precisely where tasks are performed. Affordance features of objects serve as a bridge in the functional interaction between agents and objects. However, leveraging these affordance cues to help robots achieve functional tool gras** remains unresolved. To address this, we pr…
▽ More
To enable robots to use tools, the initial step is teaching robots to employ dexterous gestures for touching specific areas precisely where tasks are performed. Affordance features of objects serve as a bridge in the functional interaction between agents and objects. However, leveraging these affordance cues to help robots achieve functional tool gras** remains unresolved. To address this, we propose a granularity-aware affordance feature extraction method for locating functional affordance areas and predicting dexterous coarse gestures. We study the intrinsic mechanisms of human tool use. On one hand, we use fine-grained affordance features of object-functional finger contact areas to locate functional affordance regions. On the other hand, we use highly activated coarse-grained affordance features in hand-object interaction regions to predict grasp gestures. Additionally, we introduce a model-based post-processing module that includes functional finger coordinate localization, finger-to-end coordinate transformation, and force feedback-based coarse-to-fine gras**. This forms a complete dexterous robotic functional gras** framework GAAF-Dex, which learns Granularity-Aware Affordances from human-object interaction for tool-based Functional gras** in Dexterous Robotics. Unlike fully-supervised methods that require extensive data annotation, we employ a weakly supervised approach to extract relevant cues from exocentric (Exo) images of hand-object interactions to supervise feature extraction in egocentric (Ego) images. We have constructed a small-scale dataset, FAH, which includes near 6K images of functional hand-object interaction Exo- and Ego images of 18 commonly used tools performing 6 tasks. Extensive experiments on the dataset demonstrate our method outperforms state-of-the-art methods. The code will be made publicly available at https://github.com/yangfan293/GAAF-DEX.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
New type of solutions for a critical Grushin-type problem with competing potentials
Authors:
Wen**g Chen,
Zexi Wang
Abstract:
In this paper, we consider a critical Grushin-type problem with double potentials. By applying the reduction argument and local Pohouzaev identities, we construct a new family of solutions to this problem, which are concentrated at points lying on the top and the bottom circles of a cylinder.
In this paper, we consider a critical Grushin-type problem with double potentials. By applying the reduction argument and local Pohouzaev identities, we construct a new family of solutions to this problem, which are concentrated at points lying on the top and the bottom circles of a cylinder.
△ Less
Submitted 29 June, 2024;
originally announced July 2024.
-
Dual-view Aware Smart Contract Vulnerability Detection for Ethereum
Authors:
Jiacheng Yao,
Maolin Wang,
Wanqi Chen,
Chengxiang **,
Jiajun Zhou,
Shanqing Yu,
Qi Xuan
Abstract:
The wide application of Ethereum technology has brought technological innovation to traditional industries. As one of Ethereum's core applications, smart contracts utilize diverse contract codes to meet various functional needs and have gained widespread use. However, the non-tamperability of smart contracts, coupled with vulnerabilities caused by natural flaws or human errors, has brought unprece…
▽ More
The wide application of Ethereum technology has brought technological innovation to traditional industries. As one of Ethereum's core applications, smart contracts utilize diverse contract codes to meet various functional needs and have gained widespread use. However, the non-tamperability of smart contracts, coupled with vulnerabilities caused by natural flaws or human errors, has brought unprecedented challenges to blockchain security. Therefore, in order to ensure the healthy development of blockchain technology and the stability of the blockchain community, it is particularly important to study the vulnerability detection techniques for smart contracts. In this paper, we propose a Dual-view Aware Smart Contract Vulnerability Detection Framework named DVDet. The framework initially converts the source code and bytecode of smart contracts into weighted graphs and control flow sequences, capturing potential risk features from these two perspectives and integrating them for analysis, ultimately achieving effective contract vulnerability detection. Comprehensive experiments on the Ethereum dataset show that our method outperforms others in detecting vulnerabilities.
△ Less
Submitted 29 June, 2024;
originally announced July 2024.
-
Next-to-leading-order QCD corrections to nucleon Dirac form factors
Authors:
Wen Chen,
Feng Feng,
Yu Jia
Abstract:
The leading-order perturbative QCD (pQCD) predictions to nucleon electromagnetic form factors were first made in late 70s. Nearly half a century later, we accomplish the calculation of the next-to-leading-order (NLO) QCD corrections to proton and neutron's Dirac form factors at large momentum transfer, to the leading-twist accuracy in collinear factorization. The effect of NLO perturbative correct…
▽ More
The leading-order perturbative QCD (pQCD) predictions to nucleon electromagnetic form factors were first made in late 70s. Nearly half a century later, we accomplish the calculation of the next-to-leading-order (NLO) QCD corrections to proton and neutron's Dirac form factors at large momentum transfer, to the leading-twist accuracy in collinear factorization. The effect of NLO perturbative corrections turns out to be positive and substantial. Confronting our state-of-the-art pQCD predictions with the available data for nucleon Dirac form factors, in both space-like and time-like regions, imposes a stringent test on the validity of the parameterized forms of nucleon leading-twist light-cone-distribution amplitudes (LCDAs), which have been predicted by a class of QCD sum rules-based models and lattice QCD simulation.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
YuLan: An Open-source Large Language Model
Authors:
Yutao Zhu,
Kun Zhou,
Kelong Mao,
Wentong Chen,
Yiding Sun,
Zhipeng Chen,
Qian Cao,
Yihan Wu,
Yushuo Chen,
Feng Wang,
Lei Zhang,
Junyi Li,
Xiaolei Wang,
Lei Wang,
Beichen Zhang,
Zican Dong,
Xiaoxue Cheng,
Yuhan Chen,
Xinyu Tang,
Yupeng Hou,
Qiangqiang Ren,
Xincheng Pang,
Shufang Xie,
Wayne Xin Zhao,
Zhicheng Dou
, et al. (13 additional authors not shown)
Abstract:
Large language models (LLMs) have become the foundation of many applications, leveraging their extensive capabilities in processing and understanding natural language. While many open-source LLMs have been released with technical reports, the lack of training details hinders further research and development. This paper presents the development of YuLan, a series of open-source LLMs with $12$ billi…
▽ More
Large language models (LLMs) have become the foundation of many applications, leveraging their extensive capabilities in processing and understanding natural language. While many open-source LLMs have been released with technical reports, the lack of training details hinders further research and development. This paper presents the development of YuLan, a series of open-source LLMs with $12$ billion parameters. The base model of YuLan is pre-trained on approximately $1.7$T tokens derived from a diverse corpus, including massive English, Chinese, and multilingual texts. We design a three-stage pre-training method to enhance YuLan's overall capabilities. Subsequent phases of training incorporate instruction-tuning and human alignment, employing a substantial volume of high-quality synthesized data. To facilitate the learning of complex and long-tail knowledge, we devise a curriculum-learning framework throughout across these stages, which helps LLMs learn knowledge in an easy-to-hard manner. YuLan's training is finished on Jan, 2024 and has achieved performance on par with state-of-the-art LLMs across various English and Chinese benchmarks. This paper outlines a comprehensive technical roadmap for develo** LLMs from scratch. Our model and codes are available at https://github.com/RUC-GSAI/YuLan-Chat.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
Estimating Long-term Heterogeneous Dose-response Curve: Generalization Bound Leveraging Optimal Transport Weights
Authors:
Zeqin Yang,
Weilin Chen,
Ruichu Cai,
Yuguang Yan,
Zhifeng Hao,
Zhipeng Yu,
Zhichao Zou,
Zhen Peng,
Jiecheng Guo
Abstract:
Long-term causal effect estimation is a significant but challenging problem in many applications. Existing methods rely on ideal assumptions to estimate long-term average effects, e.g., no unobserved confounders or a binary treatment,while in numerous real-world applications, these assumptions could be violated and average effects are unable to provide individual-level suggestions.In this paper,we…
▽ More
Long-term causal effect estimation is a significant but challenging problem in many applications. Existing methods rely on ideal assumptions to estimate long-term average effects, e.g., no unobserved confounders or a binary treatment,while in numerous real-world applications, these assumptions could be violated and average effects are unable to provide individual-level suggestions.In this paper,we address a more general problem of estimating the long-term heterogeneous dose-response curve (HDRC) while accounting for unobserved confounders. Specifically, to remove unobserved confounding in observational data, we introduce an optimal transport weighting framework to align the observational data to the experimental data with theoretical guarantees. Furthermore,to accurately predict the heterogeneous effects of continuous treatment, we establish a generalization bound on counterfactual prediction error by leveraging the reweighted distribution induced by optimal transport. Finally, we develop an HDRC estimator building upon the above theoretical foundations. Extensive experimental studies conducted on multiple synthetic and semi-synthetic datasets demonstrate the effectiveness of our proposed method.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
STBench: Assessing the Ability of Large Language Models in Spatio-Temporal Analysis
Authors:
Wenbin Li,
Di Yao,
Ruibo Zhao,
Wenjie Chen,
Zijie Xu,
Chengxue Luo,
Chang Gong,
Quanliang **g,
Haining Tan,
**g** Bi
Abstract:
The rapid evolution of large language models (LLMs) holds promise for reforming the methodology of spatio-temporal data mining. However, current works for evaluating the spatio-temporal understanding capability of LLMs are somewhat limited and biased. These works either fail to incorporate the latest language models or only focus on assessing the memorized spatio-temporal knowledge. To address thi…
▽ More
The rapid evolution of large language models (LLMs) holds promise for reforming the methodology of spatio-temporal data mining. However, current works for evaluating the spatio-temporal understanding capability of LLMs are somewhat limited and biased. These works either fail to incorporate the latest language models or only focus on assessing the memorized spatio-temporal knowledge. To address this gap, this paper dissects LLMs' capability of spatio-temporal data into four distinct dimensions: knowledge comprehension, spatio-temporal reasoning, accurate computation, and downstream applications. We curate several natural language question-answer tasks for each category and build the benchmark dataset, namely STBench, containing 13 distinct tasks and over 60,000 QA pairs. Moreover, we have assessed the capabilities of 13 LLMs, such as GPT-4o, Gemma and Mistral. Experimental results reveal that existing LLMs show remarkable performance on knowledge comprehension and spatio-temporal reasoning tasks, with potential for further enhancement on other tasks through in-context learning, chain-of-though prompting, and fine-tuning. The code and datasets of STBench are released on https://github.com/LwbXc/STBench.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Generalized Averaging Method for Power Electronics Modeling from DC to above Half the Switching Frequency
Authors:
Hongchang Li,
Kang** Wang,
**gyang Fang,
Wenjie Chen,
Xu Yang
Abstract:
Modeling power electronic converters at frequencies close to or above half the switching frequency has been difficult due to the time-variant and discontinuous switching actions. This paper uses the properties of moving Fourier coefficients to develop the generalized averaging method, breaking though the limit of half the switching frequency. The paper also proposes the generalized average model f…
▽ More
Modeling power electronic converters at frequencies close to or above half the switching frequency has been difficult due to the time-variant and discontinuous switching actions. This paper uses the properties of moving Fourier coefficients to develop the generalized averaging method, breaking though the limit of half the switching frequency. The paper also proposes the generalized average model for various switching signals, including pulse-width modulation (PWM), phase-shift modulation, pulse-frequency modulation (PFM), and state-dependent switching signals, so that circuits and modulators/controllers can be modeled separately and combined flexibly. Using the Laplace transform of moving Fourier coefficients, the coupling of signals and their sidebands at different frequencies is clearly described as the coupling of moving Fourier coefficients at the same frequency in a linear time-invariant system framework. The modeling method is applied to a PWM controlled boost converter, a V2 constant on-time controlled buck converter, and a PFM controlled LLC converter, for demonstration and validation. Experimental results of the converters in different operating modes show that the proposed models have higher accuracy than exiting models, especially in the frequency range close to or above half the switching frequency. The developed method can be applied to almost all types of power electronic converters.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing
Authors:
Yunlong Zhao,
Xiaoheng Deng,
Yi**g Liu,
Xinjun Pei,
Jiazhi Xia,
Wei Chen
Abstract:
Model stealing (MS) involves querying and observing the output of a machine learning model to steal its capabilities. The quality of queried data is crucial, yet obtaining a large amount of real data for MS is often challenging. Recent works have reduced reliance on real data by using generative models. However, when high-dimensional query data is required, these methods are impractical due to the…
▽ More
Model stealing (MS) involves querying and observing the output of a machine learning model to steal its capabilities. The quality of queried data is crucial, yet obtaining a large amount of real data for MS is often challenging. Recent works have reduced reliance on real data by using generative models. However, when high-dimensional query data is required, these methods are impractical due to the high costs of querying and the risk of model collapse. In this work, we propose using sample gradients (SG) to enhance the utility of each real sample, as SG provides crucial guidance on the decision boundaries of the victim model. However, utilizing SG in the model stealing scenario faces two challenges: 1. Pixel-level gradient estimation requires extensive query volume and is susceptible to defenses. 2. The estimation of sample gradients has a significant variance. This paper proposes Superpixel Sample Gradient stealing (SPSG) for model stealing under the constraint of limited real samples. With the basic idea of imitating the victim model's low-variance patch-level gradients instead of pixel-level gradients, SPSG achieves efficient sample gradient estimation through two steps. First, we perform patch-wise perturbations on query images to estimate the average gradient in different regions of the image. Then, we filter the gradients through a threshold strategy to reduce variance. Exhaustive experiments demonstrate that, with the same number of real samples, SPSG achieves accuracy, agreements, and adversarial success rate significantly surpassing the current state-of-the-art MS methods. Codes are available at https://github.com/zyl123456aB/SPSG_attack.
△ Less
Submitted 18 May, 2024;
originally announced June 2024.
-
VideoQA-SC: Adaptive Semantic Communication for Video Question Answering
Authors:
Jiangyuan Guo,
Wei Chen,
Yuxuan Sun,
Jialong Xu,
Bo Ai
Abstract:
Although semantic communication (SC) has shown its potential in efficiently transmitting multi-modal data such as text, speeches and images, SC for videos has focused primarily on pixel-level reconstruction. However, these SC systems may be suboptimal for downstream intelligent tasks. Moreover, SC systems without pixel-level video reconstruction present advantages by achieving higher bandwidth eff…
▽ More
Although semantic communication (SC) has shown its potential in efficiently transmitting multi-modal data such as text, speeches and images, SC for videos has focused primarily on pixel-level reconstruction. However, these SC systems may be suboptimal for downstream intelligent tasks. Moreover, SC systems without pixel-level video reconstruction present advantages by achieving higher bandwidth efficiency and real-time performance of various intelligent tasks. The difficulty in such system design lies in the extraction of task-related compact semantic representations and their accurate delivery over noisy channels. In this paper, we propose an end-to-end SC system for video question answering (VideoQA) tasks called VideoQA-SC. Our goal is to accomplish VideoQA tasks directly based on video semantics over noisy or fading wireless channels, bypassing the need for video reconstruction at the receiver. To this end, we develop a spatiotemporal semantic encoder for effective video semantic extraction, and a learning-based bandwidth-adaptive deep joint source-channel coding (DJSCC) scheme for efficient and robust video semantic transmission. Experiments demonstrate that VideoQA-SC outperforms traditional and advanced DJSCC-based SC systems that rely on video reconstruction at the receiver under a wide range of channel conditions and bandwidth constraints. In particular, when the signal-to-noise ratio is low, VideoQA-SC can improve the answer accuracy by 5.17% while saving almost 99.5% of the bandwidth at the same time, compared with the advanced DJSCC-based SC system. Our results show the great potential of task-oriented SC system design for video applications.
△ Less
Submitted 17 May, 2024;
originally announced June 2024.
-
Spatial-temporal Hierarchical Reinforcement Learning for Interpretable Pathology Image Super-Resolution
Authors:
Wenting Chen,
Jie Liu,
Tommy W. S. Chow,
Yixuan Yuan
Abstract:
Pathology image are essential for accurately interpreting lesion cells in cytopathology screening, but acquiring high-resolution digital slides requires specialized equipment and long scanning times. Though super-resolution (SR) techniques can alleviate this problem, existing deep learning models recover pathology image in a black-box manner, which can lead to untruthful biological details and mis…
▽ More
Pathology image are essential for accurately interpreting lesion cells in cytopathology screening, but acquiring high-resolution digital slides requires specialized equipment and long scanning times. Though super-resolution (SR) techniques can alleviate this problem, existing deep learning models recover pathology image in a black-box manner, which can lead to untruthful biological details and misdiagnosis. Additionally, current methods allocate the same computational resources to recover each pixel of pathology image, leading to the sub-optimal recovery issue due to the large variation of pathology image. In this paper, we propose the first hierarchical reinforcement learning framework named Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL), mainly for addressing the aforementioned issues in pathology image super-resolution problem. We reformulate the SR problem as a Markov decision process of interpretable operations and adopt the hierarchical recovery mechanism in patch level, to avoid sub-optimal recovery. Specifically, the higher-level spatial manager is proposed to pick out the most corrupted patch for the lower-level patch worker. Moreover, the higher-level temporal manager is advanced to evaluate the selected patch and determine whether the optimization should be stopped earlier, thereby avoiding the over-processed problem. Under the guidance of spatial-temporal managers, the lower-level patch worker processes the selected patch with pixel-wise interpretable actions at each time step. Experimental results on medical images degraded by different kernels show the effectiveness of STAR-RL. Furthermore, STAR-RL validates the promotion in tumor diagnosis with a large margin and shows generalizability under various degradations. The source code is available at https://github.com/CUHK-AIM-Group/STAR-RL.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
Octo-planner: On-device Language Model for Planner-Action Agents
Authors:
Wei Chen,
Zhiyuan Li,
Zhen Guo,
Yikang Shen
Abstract:
AI agents have become increasingly significant in various domains, enabling autonomous decision-making and problem-solving. To function effectively, these agents require a planning process that determines the best course of action and then executes the planned actions. In this paper, we present an efficient on-device Planner-Action framework that separates planning and action execution into two di…
▽ More
AI agents have become increasingly significant in various domains, enabling autonomous decision-making and problem-solving. To function effectively, these agents require a planning process that determines the best course of action and then executes the planned actions. In this paper, we present an efficient on-device Planner-Action framework that separates planning and action execution into two distinct components: a planner agent based on Phi-3 Mini, a 3.8 billion parameter LLM optimized for edge devices, and an action agent using the Octopus model for function execution. The planner agent first responds to user queries by decomposing tasks into a sequence of sub-steps, which are then executed by the action agent. To optimize performance on resource-constrained devices, we employ model fine-tuning instead of in-context learning, reducing computational costs and energy consumption while improving response times. Our approach involves using GPT-4 to generate diverse planning queries and responses based on available functions, with subsequent validations to ensure data quality. We fine-tune the Phi-3 Mini model on this curated dataset, achieving a 97\% success rate in our in-domain test environment. To address multi-domain planning challenges, we developed a multi-LoRA training method that merges weights from LoRAs trained on distinct function subsets. This approach enables flexible handling of complex, multi-domain queries while maintaining computational efficiency on resource-constrained devices. To support further research, we have open-sourced our model weights at \url{https://huggingface.co/NexaAIDev/octopus-planning}. For the demo, please refer to \url{https://www.nexa4ai.com/octo-planner}.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
USLC: Universal Self-Learning Control via Physical Performance Policy-Optimization Neural Network
Authors:
Yanhui Zhang,
Weifang Chen
Abstract:
This study addresses the challenge of achieving real-time Universal Self-Learning Control (USLC) in nonlinear dynamic systems with uncertain models. The proposed control method incorporates a Universal Self-Learning module, which introduces a model-free online executor-evaluator framework to enable controller adaptation in the presence of unknown disturbances. By leveraging a neural network model…
▽ More
This study addresses the challenge of achieving real-time Universal Self-Learning Control (USLC) in nonlinear dynamic systems with uncertain models. The proposed control method incorporates a Universal Self-Learning module, which introduces a model-free online executor-evaluator framework to enable controller adaptation in the presence of unknown disturbances. By leveraging a neural network model trained on historical system performance data, the controller can autonomously learn to approximate optimal performance during each learning cycle. Consequently, the controller's structural parameters are incrementally adjusted to achieve a performance threshold comparable to human-level performance. Utilizing nonlinear system stability theory, specifically in the context of three-dimensional manifold space, we demonstrate the stability of USLC in Lipschitz continuous systems. We illustrate the USLC framework numerically with two case studies: a low-order circuit system and a high-order morphing fixed-wing attitude control system. The simulation results verify the effectiveness and universality of the proposed method.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
LongIns: A Challenging Long-context Instruction-based Exam for LLMs
Authors:
Shawn Gavin,
Tuney Zheng,
Jiaheng Liu,
Quehry Que,
Noah Wang,
Jian Yang,
Chenchen Zhang,
Wenhao Huang,
Wenhu Chen,
Ge Zhang
Abstract:
The long-context capabilities of large language models (LLMs) have been a hot topic in recent years. To evaluate the performance of LLMs in different scenarios, various assessment benchmarks have emerged. However, as most of these benchmarks focus on identifying key information to answer questions, which mainly requires the retrieval ability of LLMs, these benchmarks can partially represent the re…
▽ More
The long-context capabilities of large language models (LLMs) have been a hot topic in recent years. To evaluate the performance of LLMs in different scenarios, various assessment benchmarks have emerged. However, as most of these benchmarks focus on identifying key information to answer questions, which mainly requires the retrieval ability of LLMs, these benchmarks can partially represent the reasoning performance of LLMs from large amounts of information. Meanwhile, although LLMs often claim to have context windows of 32k, 128k, 200k, or even longer, these benchmarks fail to reveal the actual supported length of these LLMs. To address these issues, we propose the LongIns benchmark dataset, a challenging long-context instruction-based exam for LLMs, which is built based on the existing instruction datasets. Specifically, in our LongIns, we introduce three evaluation settings: Global Instruction & Single Task (GIST), Local Instruction & Single Task (LIST), and Local Instruction & Multiple Tasks (LIMT). Based on LongIns, we perform comprehensive evaluations on existing LLMs and have the following important findings: (1). The top-performing GPT-4 with 128k context length performs poorly on the evaluation context window of 16k in our LongIns. (2). For the multi-hop reasoning ability of many existing LLMs, significant efforts are still needed under short context windows (less than 4k).
△ Less
Submitted 26 June, 2024; v1 submitted 25 June, 2024;
originally announced June 2024.
-
An Alternative Channel to Black Hole Low-Mass X-ray Binaries: Dynamical Friction of Dark Matter?
Authors:
Ke Qin,
Wen-Cong Chen
Abstract:
Both the anomalous magnetic braking of Ap/Bp stars and the surrounding circumbinary disk models can account for the formation of black hole (BH) low-mass X-ray binaries (LMXBs), while the simulated effective temperatures of the donor stars are significantly higher than the observed values. Therefore, the formation of BH LMXBs is not still completely understood. In this work, we diagnose whether th…
▽ More
Both the anomalous magnetic braking of Ap/Bp stars and the surrounding circumbinary disk models can account for the formation of black hole (BH) low-mass X-ray binaries (LMXBs), while the simulated effective temperatures of the donor stars are significantly higher than the observed values. Therefore, the formation of BH LMXBs is not still completely understood. In this work, we diagnose whether the dynamical friction between dark matter and the companion stars can drive BH binaries to evolve toward the observed BH LMXBs and alleviate the effective temperature problem. Assuming that there exists a density spike of dark matter around BH, the dynamical friction can produce an efficient angular momentum loss, driving BH binaries with an intermediate-mass companion star to evolve into BH LMXBs for a spike index higher than $γ= 1.58$. Our detailed stellar evolution models show that the calculated effective temperatures can match the observed value of most BH LMXBs for a spike index range of $γ= 1.7-2.1$. However, the simulated mass-transfer rates when $γ= 2.0$ and $2.1$ are too high to be consistent with the observed properties that BH LMXBs appears as soft X-ray transients. Therefore, the dynamical friction of dark matter can only alleviate the effective temperature problem of those BH LMXBs with a relatively short orbital period.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Nondegenerate Turán problems under $(t,p)$-norms
Authors:
Wanfang Chen,
Daniel Iľkovič,
Jared León,
Xizhi Liu,
Oleg Pikhurko
Abstract:
Given integers $r > t \ge 1$ and a real number $p > 0$, the $(t,p)$-norm $\left\lVert \mathcal{H} \right\rVert_{t,p}$ of an $r$-graph $\mathcal{H}$ is the sum of the $p$-th power of the degrees $d_{\mathcal{H}}(T)$ over all $t$-subsets $T \subset V(\mathcal{H})$. We conduct a systematic study of the Turán-type problem of determining $\mathrm{ex}_{t,p}(n,\mathcal{F})$, which is the maximum of…
▽ More
Given integers $r > t \ge 1$ and a real number $p > 0$, the $(t,p)$-norm $\left\lVert \mathcal{H} \right\rVert_{t,p}$ of an $r$-graph $\mathcal{H}$ is the sum of the $p$-th power of the degrees $d_{\mathcal{H}}(T)$ over all $t$-subsets $T \subset V(\mathcal{H})$. We conduct a systematic study of the Turán-type problem of determining $\mathrm{ex}_{t,p}(n,\mathcal{F})$, which is the maximum of $\left\lVert \mathcal{H} \right\rVert_{t,p}$ over all $n$-vertex $\mathcal{F}$-free $r$-graphs $\mathcal{H}$.
We establish several basic properties for the $(t,p)$-norm of $r$-graphs, enabling us to derive general theorems from the recently established framework in~\cite{CL24} that are useful for determining $\mathrm{ex}_{t,p}(n,\mathcal{F})$ and proving the corresponding stability.
We determine the asymptotic value of $\mathrm{ex}_{t,p}(n,H_{F}^{r})$ for all feasible combinations of $(r,t,p)$ and for every graph $F$ with chromatic number greater than $r$, where $H_{F}^{r}$ represents the expansion of $F$. In the case where $F$ is edge-critical and $p \ge 1$, we establish strong stability and determine the exact value of $\mathrm{ex}_{t,p}(n,H_{F}^{r})$ for all sufficiently large $n$. These results extend the seminal theorems of Erdős--Stone--Simonovits, Andrásfai--Erdős--Sós, Erdős--Simonovits, and a classical theorem of Mubayi.
For the $3$-uniform generalized triangle $F_5$, we determine the exact value of $\mathrm{ex}_{2,p}(n,F_5)$ for all $p \ge 1$ and its asymptotic value for all $p \in [1/2, 1]\cup \{k^{-1} \colon k \in 6\mathbb{N}^{+}+\{0,2\}\}$. This extends old theorems of Bollobás, Frankl--Füredi, and a recent result of Balogh--Clemen--Lidický. Our proofs utilize results on the graph inducibility problem, Steiner triple systems, and the feasible region problem introduced by Liu--Mubayi.
△ Less
Submitted 22 June, 2024;
originally announced June 2024.
-
Testing particle acceleration in blazar jets with continuous high-cadence optical polarization observations
Authors:
Ioannis Liodakis,
Sebastian Kiehlmann,
Alan P. Marscher,
Haocheng Zhang,
Dmitry Blinov,
Svetlana G. Jorstad,
Iván Agudo,
Erika Benítez,
Andrei Berdyugin,
Giacomo Bonnoli,
Carolina Casadio,
Chien-Ting Chen,
Wen-** Chen,
Steven R. Ehlert,
Juan Escudero,
Tatiana S. Grishina,
David Hiriart,
Angela Hsu,
Ryo Imazawa,
Helen E. Jermak,
**cen Jose,
Philip Kaaret,
Evgenia N. Kopatskaya,
Bhavana Lalchand,
Elena G. Larionova
, et al. (22 additional authors not shown)
Abstract:
Variability can be the pathway to understanding the physical processes in astrophysical jets, however, the high-cadence observations required to test particle acceleration models are still missing. Here we report on the first attempt to produce continuous, >24 hour polarization light curves of blazars using telescopes distributed across the globe and the rotation of the Earth to avoid the rising S…
▽ More
Variability can be the pathway to understanding the physical processes in astrophysical jets, however, the high-cadence observations required to test particle acceleration models are still missing. Here we report on the first attempt to produce continuous, >24 hour polarization light curves of blazars using telescopes distributed across the globe and the rotation of the Earth to avoid the rising Sun. Our campaign involved 16 telescopes in Asia, Europe, and North America. We observed BL Lacertae and CGRaBS J0211+1051 for a combined 685 telescope hours. We find large variations in the polarization degree and angle for both sources in sub-hour timescales as well as a ~180 degree rotation of the polarization angle in CGRaBS J0211+1051 in less than two days. We compared our high-cadence observations to Particle-In-Cell magnetic reconnection and turbulent plasma simulations. We find that although the state of the art simulation frameworks can produce a large fraction of the polarization properties, they do not account for the entirety of the observed polarization behavior in blazar jets.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
Authors:
Ziyan Jiang,
Xueguang Ma,
Wenhu Chen
Abstract:
In traditional RAG framework, the basic retrieval units are normally short. The common retrievers like DPR normally work with 100-word Wikipedia paragraphs. Such a design forces the retriever to search over a large corpus to find the `needle' unit. In contrast, the readers only need to extract answers from the short retrieved units. Such an imbalanced `heavy' retriever and `light' reader design ca…
▽ More
In traditional RAG framework, the basic retrieval units are normally short. The common retrievers like DPR normally work with 100-word Wikipedia paragraphs. Such a design forces the retriever to search over a large corpus to find the `needle' unit. In contrast, the readers only need to extract answers from the short retrieved units. Such an imbalanced `heavy' retriever and `light' reader design can lead to sub-optimal performance. In order to alleviate the imbalance, we propose a new framework LongRAG, consisting of a `long retriever' and a `long reader'. LongRAG processes the entire Wikipedia into 4K-token units, which is 30x longer than before. By increasing the unit size, we significantly reduce the total units from 22M to 700K. This significantly lowers the burden of retriever, which leads to a remarkable retrieval score: answer recall@1=71% on NQ (previously 52%) and answer recall@2=72% (previously 47%) on HotpotQA (full-wiki). Then we feed the top-k retrieved units ($\approx$ 30K tokens) to an existing long-context LLM to perform zero-shot answer extraction. Without requiring any training, LongRAG achieves an EM of 62.7% on NQ, which is the best known result. LongRAG also achieves 64.3% on HotpotQA (full-wiki), which is on par of the SoTA model. Our study offers insights into the future roadmap for combining RAG with long-context LLMs.
△ Less
Submitted 30 June, 2024; v1 submitted 21 June, 2024;
originally announced June 2024.
-
VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation
Authors:
Xuan He,
Dongfu Jiang,
Ge Zhang,
Max Ku,
Achint Soni,
Sherman Siu,
Haonan Chen,
Abhranil Chandra,
Ziyan Jiang,
Aaran Arulraj,
Kai Wang,
Quy Duc Do,
Yuansheng Ni,
Bohan Lyu,
Yaswanth Narsupalli,
Rongqi Fan,
Zhiheng Lyu,
Yuchen Lin,
Wenhu Chen
Abstract:
The recent years have witnessed great advances in video generation. However, the development of automatic video metrics is lagging significantly behind. None of the existing metric is able to provide reliable scores over generated videos. The main barrier is the lack of large-scale human-annotated dataset. In this paper, we release VideoFeedback, the first large-scale dataset containing human-prov…
▽ More
The recent years have witnessed great advances in video generation. However, the development of automatic video metrics is lagging significantly behind. None of the existing metric is able to provide reliable scores over generated videos. The main barrier is the lack of large-scale human-annotated dataset. In this paper, we release VideoFeedback, the first large-scale dataset containing human-provided multi-aspect score over 37.6K synthesized videos from 11 existing video generative models. We train VideoScore (initialized from Mantis) based on VideoFeedback to enable automatic video quality assessment. Experiments show that the Spearman correlation between VideoScore and humans can reach 77.1 on VideoFeedback-test, beating the prior best metrics by about 50 points. Further result on other held-out EvalCrafter, GenAI-Bench, and VBench show that VideoScore has consistently much higher correlation with human judges than other metrics. Due to these results, we believe VideoScore can serve as a great proxy for human raters to (1) rate different video models to track progress (2) simulate fine-grained human feedback in Reinforcement Learning with Human Feedback (RLHF) to improve current video generation models.
△ Less
Submitted 24 June, 2024; v1 submitted 21 June, 2024;
originally announced June 2024.
-
Quantum geometrical properties of topological materials
Authors:
Wei Chen
Abstract:
The momentum space of topological insulators and topological superconductors is equipped with a quantum metric defined from the overlap of neighboring valence band states or quasihole states. We investigate the quantum geometrical properties of these materials within the framework of Dirac models and differential geometry. The Ricci scalar is found to be a constant throughout the whole Brillouin z…
▽ More
The momentum space of topological insulators and topological superconductors is equipped with a quantum metric defined from the overlap of neighboring valence band states or quasihole states. We investigate the quantum geometrical properties of these materials within the framework of Dirac models and differential geometry. The Ricci scalar is found to be a constant throughout the whole Brillouin zone, and the vacuum Einstein equation is satisfied in 3D with a finite cosmological constant. For linear Dirac models, several geometrical properties are found to be independent of the band gap, including the straight line geodesic, constant volume of the curved momentum space, exponential decay form of the nonlocal topological marker, and unity Euler characteristic in 2D, indicating the peculiar yet universal quantum geometrical properties of these models.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
ICLEval: Evaluating In-Context Learning Ability of Large Language Models
Authors:
Wentong Chen,
Yankai Lin,
ZhenHao Zhou,
HongYun Huang,
Yantao Jia,
Zhao Cao,
Ji-Rong Wen
Abstract:
In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs. Evaluating the ICL ability of LLMs can enhance their utilization and deepen our understanding of how this ability is acquired at the training stage. However, existing evaluation frameworks primarily focus on language abilities and knowledge,…
▽ More
In-Context Learning (ICL) is a critical capability of Large Language Models (LLMs) as it empowers them to comprehend and reason across interconnected inputs. Evaluating the ICL ability of LLMs can enhance their utilization and deepen our understanding of how this ability is acquired at the training stage. However, existing evaluation frameworks primarily focus on language abilities and knowledge, often overlooking the assessment of ICL ability. In this work, we introduce the ICLEval benchmark to evaluate the ICL abilities of LLMs, which encompasses two key sub-abilities: exact copying and rule learning. Through the ICLEval benchmark, we demonstrate that ICL ability is universally present in different LLMs, and model size is not the sole determinant of ICL efficacy. Surprisingly, we observe that ICL abilities, particularly copying, develop early in the pretraining process and stabilize afterward. Our source codes and benchmark are released at https://github.com/yiye3/ICLEval.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
RIS-aided MIMO Beamforming: Piece-Wise Near-field Channel Model
Authors:
Weijian Chen,
Zai Yang,
Zhiqiang Wei,
Derrick Wing Kwan Ng,
Michail Matthaiou
Abstract:
This paper proposes a joint active and passive beamforming design for reconfigurable intelligent surface (RIS)-aided wireless communication systems, adopting a piece-wise near-field channel model. While a traditional near-field channel model, applied without any approximations, offers higher modeling accuracy than a far-field model, it renders the system design more sensitive to channel estimation…
▽ More
This paper proposes a joint active and passive beamforming design for reconfigurable intelligent surface (RIS)-aided wireless communication systems, adopting a piece-wise near-field channel model. While a traditional near-field channel model, applied without any approximations, offers higher modeling accuracy than a far-field model, it renders the system design more sensitive to channel estimation errors (CEEs). As a remedy, we propose to adopt a piece-wise near-field channel model that leverages the advantages of the near-field approach while enhancing its robustness against CEEs. Our study analyzes the impact of different channel models, including the traditional near-field, the proposed piece-wise near-field and far-field channel models, on the interference distribution caused by CEEs and model mismatches. Subsequently, by treating the interference as noise, we formulate a joint active and passive beamforming design problem to maximize the spectral efficiency (SE). The formulated problem is then recast as a mean squared error (MSE) minimization problem and a suboptimal algorithm is developed to iteratively update the active and passive beamforming strategies. Simulation results demonstrate that adopting the piece-wise near-field channel model leads to an improved SE compared to both the near-field and far-field models in the presence of CEEs. Furthermore, the proposed piece-wise near-field model achieves a good trade-off between modeling accuracy and system's degrees of freedom (DoF).
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
Autonomous Agents for Collaborative Task under Information Asymmetry
Authors:
Wei Liu,
Chenxi Wang,
Yifei Wang,
Zihao Xie,
Rennai Qiu,
Yufan Dang,
Zhuoyun Du,
Weize Chen,
Cheng Yang,
Chen Qian
Abstract:
Large Language Model Multi-Agent Systems (LLM-MAS) have achieved great progress in solving complex tasks. It performs communication among agents within the system to collaboratively solve tasks, under the premise of shared information. However, when agents' communication is leveraged to enhance human cooperation, a new challenge arises due to information asymmetry, since each agent can only access…
▽ More
Large Language Model Multi-Agent Systems (LLM-MAS) have achieved great progress in solving complex tasks. It performs communication among agents within the system to collaboratively solve tasks, under the premise of shared information. However, when agents' communication is leveraged to enhance human cooperation, a new challenge arises due to information asymmetry, since each agent can only access the information of its human user. Previous MAS struggle to complete tasks under this condition. To address this, we propose a new MAS paradigm termed iAgents, which denotes Informative Multi-Agent Systems. In iAgents, the human social network is mirrored in the agent network, where agents proactively exchange human information necessary for task resolution, thereby overcoming information asymmetry. iAgents employs a novel agent reasoning mechanism, InfoNav, to navigate agents' communication towards effective information exchange. Together with InfoNav, iAgents organizes human information in a mixed memory to provide agents with accurate and comprehensive information for exchange. Additionally, we introduce InformativeBench, the first benchmark tailored for evaluating LLM agents' task-solving ability under information asymmetry. Experimental results show that iAgents can collaborate within a social network of 140 individuals and 588 relationships, autonomously communicate over 30 turns, and retrieve information from nearly 70,000 messages to complete tasks within 3 minutes.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
A new flow dynamic approach for Wasserstein gradient flows
Authors:
Qing Cheng,
Qianqian Liu,
Wenbin Chen,
Jie Shen
Abstract:
We develop in this paper a new regularized flow dynamic approach to construct efficient numerical schemes for Wasserstein gradient flows in Lagrangian coordinates. Instead of approximating the Wasserstein distance which needs to solve constrained minimization problems, we reformulate the problem using the Benamou-Brenier's flow dynamic approach, leading to algorithms which only need to solve uncon…
▽ More
We develop in this paper a new regularized flow dynamic approach to construct efficient numerical schemes for Wasserstein gradient flows in Lagrangian coordinates. Instead of approximating the Wasserstein distance which needs to solve constrained minimization problems, we reformulate the problem using the Benamou-Brenier's flow dynamic approach, leading to algorithms which only need to solve unconstrained minimization problem in $L^2$ distance. Our schemes automatically inherit some essential properties of Wasserstein gradient systems such as positivity-preserving, mass conservative and energy dissipation. We present ample numerical simulations of Porous-Medium equations, Keller-Segel equations and Aggregation equations to validate the accuracy and stability of the proposed schemes. Compared to numerical schemes in Eulerian coordinates, our new schemes can capture sharp interfaces for various Wasserstein gradient flows using relatively smaller number of unknowns.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms
Authors:
Weiqin Chen,
Mark S. Squillante,
Chai Wah Wu,
Santiago Paternain
Abstract:
We devise a control-theoretic reinforcement learning approach to support direct learning of the optimal policy. We establish theoretical properties of our approach and derive an algorithm based on a specific instance of this approach. Our empirical results demonstrate the significant benefits of our approach.
We devise a control-theoretic reinforcement learning approach to support direct learning of the optimal policy. We establish theoretical properties of our approach and derive an algorithm based on a specific instance of this approach. Our empirical results demonstrate the significant benefits of our approach.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Towards Event-oriented Long Video Understanding
Authors:
Yifan Du,
Kun Zhou,
Yuqi Huo,
Yifan Li,
Wayne Xin Zhao,
Haoyu Lu,
Zijia Zhao,
Bingning Wang,
Weipeng Chen,
Ji-Rong Wen
Abstract:
With the rapid development of video Multimodal Large Language Models (MLLMs), numerous benchmarks have been proposed to assess their video understanding capability. However, due to the lack of rich events in the videos, these datasets may suffer from the short-cut bias that the answers can be deduced from a few frames, without the need to watch the entire video. To address this issue, we introduce…
▽ More
With the rapid development of video Multimodal Large Language Models (MLLMs), numerous benchmarks have been proposed to assess their video understanding capability. However, due to the lack of rich events in the videos, these datasets may suffer from the short-cut bias that the answers can be deduced from a few frames, without the need to watch the entire video. To address this issue, we introduce Event-Bench, an event-oriented long video understanding benchmark built on existing datasets and human annotations. Event-Bench includes six event-related tasks and 2,190 test instances to comprehensively evaluate video event understanding ability. Additionally, we propose Video Instruction Merging~(VIM), a cost-effective method that enhances video MLLMs using merged, event-intensive video instructions, addressing the scarcity of human-annotated, event-intensive data. Extensive experiments show that the best-performing model, GPT-4o, achieves an overall accuracy of 53.33, significantly outperforming the best open-source model by 41.42%. Leveraging an effective instruction synthesis method and an adaptive model architecture, VIM surpasses both state-of-the-art open-source models and GPT-4V on the Event-Bench. All code, data, and models are publicly available at https://github.com/RUCAIBox/Event-Bench.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective
Authors:
Yuchen Wen,
Ke** Bi,
Wei Chen,
Jiafeng Guo,
Xueqi Cheng
Abstract:
As Large Language Models (LLMs) become an important way of information seeking, there have been increasing concerns about the unethical content LLMs may generate. In this paper, we conduct a rigorous evaluation of LLMs' implicit bias towards certain groups by attacking them with carefully crafted instructions to elicit biased responses. Our attack methodology is inspired by psychometric principles…
▽ More
As Large Language Models (LLMs) become an important way of information seeking, there have been increasing concerns about the unethical content LLMs may generate. In this paper, we conduct a rigorous evaluation of LLMs' implicit bias towards certain groups by attacking them with carefully crafted instructions to elicit biased responses. Our attack methodology is inspired by psychometric principles in cognitive and social psychology. We propose three attack approaches, i.e., Disguise, Deception, and Teaching, based on which we built evaluation datasets for four common bias types. Each prompt attack has bilingual versions. Extensive evaluation of representative LLMs shows that 1) all three attack methods work effectively, especially the Deception attacks; 2) GLM-3 performs the best in defending our attacks, compared to GPT-3.5 and GPT-4; 3) LLMs could output content of other bias types when being taught with one type of bias. Our methodology provides a rigorous and effective way of evaluating LLMs' implicit bias and will benefit the assessments of LLMs' potential ethical risks.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Soft-QMIX: Integrating Maximum Entropy For Monotonic Value Function Factorization
Authors:
Wentse Chen,
Shiyu Huang,
Jeff Schneider
Abstract:
Multi-agent reinforcement learning (MARL) tasks often utilize a centralized training with decentralized execution (CTDE) framework. QMIX is a successful CTDE method that learns a credit assignment function to derive local value functions from a global value function, defining a deterministic local policy. However, QMIX is hindered by its poor exploration strategy. While maximum entropy reinforceme…
▽ More
Multi-agent reinforcement learning (MARL) tasks often utilize a centralized training with decentralized execution (CTDE) framework. QMIX is a successful CTDE method that learns a credit assignment function to derive local value functions from a global value function, defining a deterministic local policy. However, QMIX is hindered by its poor exploration strategy. While maximum entropy reinforcement learning (RL) promotes better exploration through stochastic policies, QMIX's process of credit assignment conflicts with the maximum entropy objective and the decentralized execution requirement, making it unsuitable for maximum entropy RL. In this paper, we propose an enhancement to QMIX by incorporating an additional local Q-value learning method within the maximum entropy RL framework. Our approach constrains the local Q-value estimates to maintain the correct ordering of all actions. Due to the monotonicity of the QMIX value function, these updates ensure that locally optimal actions align with globally optimal actions. We theoretically prove the monotonic improvement and convergence of our method to an optimal solution. Experimentally, we validate our algorithm in matrix games, Multi-Agent Particle Environment and demonstrate state-of-the-art performance in SMAC-v2.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
PIN: A Knowledge-Intensive Dataset for Paired and Interleaved Multimodal Documents
Authors:
Junjie Wang,
Yin Zhang,
Yatai Ji,
Yuxiang Zhang,
Chunyang Jiang,
Yubo Wang,
Kang Zhu,
Zekun Wang,
Tiezhen Wang,
Wenhao Huang,
Jie Fu,
Bei Chen,
Qunshu Lin,
Minghao Liu,
Ge Zhang,
Wenhu Chen
Abstract:
Recent advancements in Large Multimodal Models (LMMs) have leveraged extensive multimodal datasets to enhance capabilities in complex knowledge-driven tasks. However, persistent challenges in perceptual and reasoning errors limit their efficacy, particularly in interpreting intricate visual data and deducing multimodal relationships. Addressing these issues, we introduce a novel dataset format, PI…
▽ More
Recent advancements in Large Multimodal Models (LMMs) have leveraged extensive multimodal datasets to enhance capabilities in complex knowledge-driven tasks. However, persistent challenges in perceptual and reasoning errors limit their efficacy, particularly in interpreting intricate visual data and deducing multimodal relationships. Addressing these issues, we introduce a novel dataset format, PIN (Paired and INterleaved multimodal documents), designed to significantly improve both the depth and breadth of multimodal training. The PIN format is built on three foundational principles: knowledge intensity, scalability, and support for diverse training modalities. This innovative format combines markdown files and comprehensive images to enrich training data with a dense knowledge structure and versatile training strategies. We present PIN-14M, an open-source dataset comprising 14 million samples derived from a diverse range of Chinese and English sources, tailored to include complex web and scientific content. This dataset is constructed meticulously to ensure data quality and ethical integrity, aiming to facilitate advanced training strategies and improve model robustness against common multimodal training pitfalls. Our initial results, forming the basis of this technical report, suggest significant potential for the PIN format in refining LMM performance, with plans for future expansions and detailed evaluations of its impact on model capabilities.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Advancing UWF-SLO Vessel Segmentation with Source-Free Active Domain Adaptation and a Novel Multi-Center Dataset
Authors:
Hongqiu Wang,
Xiangde Luo,
Wu Chen,
Qingqing Tang,
Mei Xin,
Qiong Wang,
Lei Zhu
Abstract:
Accurate vessel segmentation in Ultra-Wide-Field Scanning Laser Ophthalmoscopy (UWF-SLO) images is crucial for diagnosing retinal diseases. Although recent techniques have shown encouraging outcomes in vessel segmentation, models trained on one medical dataset often underperform on others due to domain shifts. Meanwhile, manually labeling high-resolution UWF-SLO images is an extremely challenging,…
▽ More
Accurate vessel segmentation in Ultra-Wide-Field Scanning Laser Ophthalmoscopy (UWF-SLO) images is crucial for diagnosing retinal diseases. Although recent techniques have shown encouraging outcomes in vessel segmentation, models trained on one medical dataset often underperform on others due to domain shifts. Meanwhile, manually labeling high-resolution UWF-SLO images is an extremely challenging, time-consuming and expensive task. In response, this study introduces a pioneering framework that leverages a patch-based active domain adaptation approach. By actively recommending a few valuable image patches by the devised Cascade Uncertainty-Predominance (CUP) selection strategy for labeling and model-finetuning, our method significantly improves the accuracy of UWF-SLO vessel segmentation across diverse medical centers. In addition, we annotate and construct the first Multi-center UWF-SLO Vessel Segmentation (MU-VS) dataset to promote this topic research, comprising data from multiple institutions. This dataset serves as a valuable resource for cross-center evaluation, verifying the effectiveness and robustness of our approach. Experimental results demonstrate that our approach surpasses existing domain adaptation and active learning methods, considerably reducing the gap between the Upper and Lower bounds with minimal annotations, highlighting our method's practical clinical value. We will release our dataset and code to facilitate relevant research: https://github.com/whq-xxh/SFADA-UWF-SLO.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
InstructRAG: Instructing Retrieval-Augmented Generation with Explicit Denoising
Authors:
Zhepei Wei,
Wei-Lin Chen,
Yu Meng
Abstract:
Retrieval-augmented generation (RAG) has shown promising potential to enhance the accuracy and factuality of language models (LMs). However, imperfect retrievers or noisy corpora can introduce misleading or even erroneous information to the retrieved contents, posing a significant challenge to the generation quality. Existing RAG methods typically address this challenge by directly predicting fina…
▽ More
Retrieval-augmented generation (RAG) has shown promising potential to enhance the accuracy and factuality of language models (LMs). However, imperfect retrievers or noisy corpora can introduce misleading or even erroneous information to the retrieved contents, posing a significant challenge to the generation quality. Existing RAG methods typically address this challenge by directly predicting final answers despite potentially noisy inputs, resulting in an implicit denoising process that is difficult to interpret and verify. On the other hand, the acquisition of explicit denoising supervision is often costly, involving significant human efforts. In this work, we propose InstructRAG, where LMs explicitly learn the denoising process through self-synthesized rationales -- First, we instruct the LM to explain how the ground-truth answer is derived from retrieved documents. Then, these rationales can be used either as demonstrations for in-context learning of explicit denoising or as supervised fine-tuning data to train the model. Compared to standard RAG approaches, InstructRAG requires no additional supervision, allows for easier verification of the predicted answers, and effectively improves generation accuracy. Experiments show InstructRAG consistently outperforms existing RAG methods in both training-free and trainable scenarios, achieving a relative improvement of 8.3% over the best baseline method on average across five knowledge-intensive benchmarks. Extensive analysis indicates that InstructRAG scales well with increased numbers of retrieved documents and consistently exhibits robust denoising ability even in out-of-domain datasets, demonstrating strong generalizability.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Voltage-controlled non-axisymmetric vibrations of soft electro-active tubes with strain-stiffening effect
Authors:
F. Zhu,
B. Wu,
M. Destrade,
H. Wang,
R. Bao,
W. Chen
Abstract:
Material properties of soft electro-active (SEA) structures are significantly sensitive to external electro-mechanical biasing fields (such as pre-stretch and electric stimuli), which generate remarkable knock-on effects on their dynamic characteristics. In this work, we analyze the electrostatically tunable non-axisymmetric vibrations of an incompressible SEA cylindrical tube under the combinatio…
▽ More
Material properties of soft electro-active (SEA) structures are significantly sensitive to external electro-mechanical biasing fields (such as pre-stretch and electric stimuli), which generate remarkable knock-on effects on their dynamic characteristics. In this work, we analyze the electrostatically tunable non-axisymmetric vibrations of an incompressible SEA cylindrical tube under the combination of a radially applied electric voltage and an axial pre-stretch. Following the theory of nonlinear electro-elasticity and the associated linearized theory for superimposed perturbations, we derive the nonlinear static response of the SEA tube to the inhomogeneous biasing fields for the Gent ideal dielectric model. Using the State Space Method, we efficiently obtain the frequency equations for voltage-controlled small-amplitude three-dimensional non-axisymmetric vibrations, covering a wide range of behaviors, from the purely radial breathing mode to torsional modes, axisymmetric longitudinal modes, and prismatic diffuse modes. We also perform an exhaustive numerical analysis to validate the proposed approach compared with the conventional displacement method, as well as to elucidate the influences of the applied voltage, axial pre-stretch, and strain-stiffening effect on the nonlinear static response and vibration behaviors of the SEA tube. The present study clearly indicates that manipulating electro-mechanical biasing fields is a feasible way to tune the small-amplitude vibration characteristics of an SEA tube. The results should benefit experimental work on, and design of, voltage-controlled resonant devices made of SEA tubes.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
AntibodyFlow: Normalizing Flow Model for Designing Antibody Complementarity-Determining Regions
Authors:
Bohao Xu,
Yanbo Wang,
Wenyu Chen,
Shimin Shan
Abstract:
Therapeutic antibodies have been extensively studied in drug discovery and development in the past decades. Antibodies are specialized protective proteins that bind to antigens in a lock-to-key manner. The binding strength/affinity between an antibody and a specific antigen is heavily determined by the complementarity-determining regions (CDRs) on the antibodies. Existing machine learning methods…
▽ More
Therapeutic antibodies have been extensively studied in drug discovery and development in the past decades. Antibodies are specialized protective proteins that bind to antigens in a lock-to-key manner. The binding strength/affinity between an antibody and a specific antigen is heavily determined by the complementarity-determining regions (CDRs) on the antibodies. Existing machine learning methods cast in silico development of CDRs as either sequence or 3D graph (with a single chain) generation tasks and have achieved initial success. However, with CDR loops having specific geometry shapes, learning the 3D geometric structures of CDRs remains a challenge. To address this issue, we propose AntibodyFlow, a 3D flow model to design antibody CDR loops. Specifically, AntibodyFlow first constructs the distance matrix, then predicts amino acids conditioned on the distance matrix. Also, AntibodyFlow conducts constraint learning and constrained generation to ensure valid 3D structures. Experimental results indicate that AntibodyFlow outperforms the best baseline consistently with up to 16.0% relative improvement in validity rate and 24.3% relative reduction in geometric graph level error (root mean square deviation, RMSD).
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
Bridging the Gap: Integrating Pre-trained Speech Enhancement and Recognition Models for Robust Speech Recognition
Authors:
Kuan-Chen Wang,
You-** Li,
Wei-Lun Chen,
Yu-Wen Chen,
Yi-Ching Wang,
**-Cheng Yeh,
Chao Zhang,
Yu Tsao
Abstract:
Noise robustness is critical when applying automatic speech recognition (ASR) in real-world scenarios. One solution involves the used of speech enhancement (SE) models as the front end of ASR. However, neural network-based (NN-based) SE often introduces artifacts into the enhanced signals and harms ASR performance, particularly when SE and ASR are independently trained. Therefore, this study intro…
▽ More
Noise robustness is critical when applying automatic speech recognition (ASR) in real-world scenarios. One solution involves the used of speech enhancement (SE) models as the front end of ASR. However, neural network-based (NN-based) SE often introduces artifacts into the enhanced signals and harms ASR performance, particularly when SE and ASR are independently trained. Therefore, this study introduces a simple yet effective SE post-processing technique to address the gap between various pre-trained SE and ASR models. A bridge module, which is a lightweight NN, is proposed to evaluate the signal-level information of the speech signal. Subsequently, using the signal-level information, the observation addition technique is applied to effectively reduce the shortcomings of SE. The experimental results demonstrate the success of our method in integrating diverse pre-trained SE and ASR models, considerably boosting the ASR robustness. Crucially, no prior knowledge of the ASR or speech contents is required during the training or inference stages. Moreover, the effectiveness of this approach extends to different datasets without necessitating the fine-tuning of the bridge module, ensuring efficiency and improved generalization.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
Authors:
Shuting Wang,
Xin Yu,
Mang Wang,
Weipeng Chen,
Yutao Zhu,
Zhicheng Dou
Abstract:
Retrieval-augmented generation (RAG) effectively addresses issues of static knowledge and hallucination in large language models. Existing studies mostly focus on question scenarios with clear user intents and concise answers. However, it is prevalent that users issue broad, open-ended queries with diverse sub-intents, for which they desire rich and long-form answers covering multiple relevant asp…
▽ More
Retrieval-augmented generation (RAG) effectively addresses issues of static knowledge and hallucination in large language models. Existing studies mostly focus on question scenarios with clear user intents and concise answers. However, it is prevalent that users issue broad, open-ended queries with diverse sub-intents, for which they desire rich and long-form answers covering multiple relevant aspects. To tackle this important yet underexplored problem, we propose a novel RAG framework, namely RichRAG. It includes a sub-aspect explorer to identify potential sub-aspects of input questions, a multi-faceted retriever to build a candidate pool of diverse external documents related to these sub-aspects, and a generative list-wise ranker, which is a key module to provide the top-k most valuable documents for the final generator. These ranked documents sufficiently cover various query aspects and are aware of the generator's preferences, hence incentivizing it to produce rich and comprehensive responses for users. The training of our ranker involves a supervised fine-tuning stage to ensure the basic coverage of documents, and a reinforcement learning stage to align downstream LLM's preferences to the ranking of documents. Experimental results on two publicly available datasets prove that our framework effectively and efficiently provides comprehensive and satisfying responses to users.
△ Less
Submitted 21 June, 2024; v1 submitted 18 June, 2024;
originally announced June 2024.
-
Schwarzschild-f(R) Black Holes and the Absence of Black Hole Bomb Effect
Authors:
Wen-Xiang Chen
Abstract:
In this paper, we analytically study the superradiant stability of Schwarzschild-f(R) black holes under charged massive scalar perturbation. Using an analytic method based on Descartes' rule of signs, we extend previous methods developed for higher-dimensional Reissner-Nordstrom black holes to the Schwarzschild-f(R) black hole cases. Our results show that Schwarzschild-f(R) black holes are superra…
▽ More
In this paper, we analytically study the superradiant stability of Schwarzschild-f(R) black holes under charged massive scalar perturbation. Using an analytic method based on Descartes' rule of signs, we extend previous methods developed for higher-dimensional Reissner-Nordstrom black holes to the Schwarzschild-f(R) black hole cases. Our results show that Schwarzschild-f(R) black holes are superradiantly stable under charged massive scalar perturbation, implying the absence of the black hole bomb effect.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
PDSS: A Privacy-Preserving Framework for Step-by-Step Distillation of Large Language Models
Authors:
Tao Fan,
Yan Kang,
Wei**g Chen,
Hanlin Gu,
Yuanfeng Song,
Lixin Fan,
Kai Chen,
Qiang Yang
Abstract:
In the context of real-world applications, leveraging large language models (LLMs) for domain-specific tasks often faces two major challenges: domain-specific knowledge privacy and constrained resources. To address these issues, we propose PDSS, a privacy-preserving framework for step-by-step distillation of LLMs. PDSS works on a server-client architecture, wherein client transmits perturbed promp…
▽ More
In the context of real-world applications, leveraging large language models (LLMs) for domain-specific tasks often faces two major challenges: domain-specific knowledge privacy and constrained resources. To address these issues, we propose PDSS, a privacy-preserving framework for step-by-step distillation of LLMs. PDSS works on a server-client architecture, wherein client transmits perturbed prompts to the server's LLM for rationale generation. The generated rationales are then decoded by the client and used to enrich the training of task-specific small language model(SLM) within a multi-task learning paradigm. PDSS introduces two privacy protection strategies: the Exponential Mechanism Strategy and the Encoder-Decoder Strategy, balancing prompt privacy and rationale usability. Experiments demonstrate the effectiveness of PDSS in various text generation tasks, enabling the training of task-specific SLM with enhanced performance while prioritizing data privacy protection.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models
Authors:
Jie Chen,
Yupeng Zhang,
Bingning Wang,
Wayne Xin Zhao,
Ji-Rong Wen,
Weipeng Chen
Abstract:
Synthetic data has been proposed as a solution to address the issue of high-quality data scarcity in the training of large language models (LLMs). Studies have shown that synthetic data can effectively improve the performance of LLMs on downstream benchmarks. However, despite its potential benefits, our analysis suggests that there may be inherent flaws in synthetic data. The uniform format of syn…
▽ More
Synthetic data has been proposed as a solution to address the issue of high-quality data scarcity in the training of large language models (LLMs). Studies have shown that synthetic data can effectively improve the performance of LLMs on downstream benchmarks. However, despite its potential benefits, our analysis suggests that there may be inherent flaws in synthetic data. The uniform format of synthetic data can lead to pattern overfitting and cause significant shifts in the output distribution, thereby reducing the model's instruction-following capabilities. Our work delves into these specific flaws associated with question-answer (Q-A) pairs, a prevalent type of synthetic data, and presents a method based on unlearning techniques to mitigate these flaws. The empirical results demonstrate the effectiveness of our approach, which can reverse the instruction-following issues caused by pattern overfitting without compromising performance on benchmarks at relatively low cost. Our work has yielded key insights into the effective use of synthetic data, aiming to promote more robust and efficient LLM training.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke
Authors:
**gxi Xu,
Runsheng Wang,
Siqi Shang,
Ava Chen,
Lauren Winterbottom,
To-Liang Hsu,
Wenxi Chen,
Khondoker Ahmed,
Pedro Leandro La Rotta,
Xinyue Zhu,
Dawn M. Nilsen,
Joel Stein,
Matei Ciocarlie
Abstract:
Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection from impaired subjects. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train i…
▽ More
Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection from impaired subjects. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this paper, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor. Videos and additional information can be found at https://jxu.ai/chatemg.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
Unregulated Divergences of Feynman Integrals
Authors:
Wen Chen
Abstract:
Feynman integrals can be expanded asymptotically with respect to some small parameters at the integrand level, a technique known as the expansion by regions. A naive expansion by regions may break down due to divergences not regulated by the spacetime dimension, exemplified by the rapidity divergences. A criterion to identify unregulated divergences is provided in this article. The analysis is con…
▽ More
Feynman integrals can be expanded asymptotically with respect to some small parameters at the integrand level, a technique known as the expansion by regions. A naive expansion by regions may break down due to divergences not regulated by the spacetime dimension, exemplified by the rapidity divergences. A criterion to identify unregulated divergences is provided in this article. The analysis is conducted using both parametric and Mellin-Barnes representations, leading to a consistent conclusion. Based on this analysis, it is proven that {\it the presence of unregulated divergences implies the degeneracies of regions}, which may help to explain phenomena like the collinear anomaly.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
SWCF-Net: Similarity-weighted Convolution and Local-global Fusion for Efficient Large-scale Point Cloud Semantic Segmentation
Authors:
Zhenchao Lin,
Li He,
Hongqiang Yang,
Xiaoqun Sun,
Cuo** Zhang,
Weinan Chen,
Yisheng Guan,
Hong Zhang
Abstract:
Large-scale point cloud consists of a multitude of individual objects, thereby encompassing rich structural and underlying semantic contextual information, resulting in a challenging problem in efficiently segmenting a point cloud. Most existing researches mainly focus on capturing intricate local features without giving due consideration to global ones, thus failing to leverage semantic context.…
▽ More
Large-scale point cloud consists of a multitude of individual objects, thereby encompassing rich structural and underlying semantic contextual information, resulting in a challenging problem in efficiently segmenting a point cloud. Most existing researches mainly focus on capturing intricate local features without giving due consideration to global ones, thus failing to leverage semantic context. In this paper, we propose a Similarity-Weighted Convolution and local-global Fusion Network, named SWCF-Net, which takes into account both local and global features. We propose a Similarity-Weighted Convolution (SWConv) to effectively extract local features, where similarity weights are incorporated into the convolution operation to enhance the generalization capabilities. Then, we employ a downsampling operation on the K and V channels within the attention module, thereby reducing the quadratic complexity to linear, enabling the Transformer to deal with large-scale point clouds. At last, orthogonal components are extracted in the global features and then aggregated with local features, thereby eliminating redundant information between local and global features and consequently promoting efficiency. We evaluate SWCF-Net on large-scale outdoor datasets SemanticKITTI and Toronto3D. Our experimental results demonstrate the effectiveness of the proposed network. Our method achieves a competitive result with less computational cost, and is able to handle large-scale point clouds efficiently.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic
Authors:
Yuyan Zhou,
Liang Song,
Bingning Wang,
Weipeng Chen
Abstract:
The advent of large language models (LLMs) like GPT-4 has catalyzed the exploration of multi-task learning (MTL), in which a single model demonstrates proficiency across diverse tasks. Task arithmetic has emerged as a cost-effective approach for MTL. It enables performance enhancement across multiple tasks by adding their corresponding task vectors to a pre-trained model. However, the current lack…
▽ More
The advent of large language models (LLMs) like GPT-4 has catalyzed the exploration of multi-task learning (MTL), in which a single model demonstrates proficiency across diverse tasks. Task arithmetic has emerged as a cost-effective approach for MTL. It enables performance enhancement across multiple tasks by adding their corresponding task vectors to a pre-trained model. However, the current lack of a method that can simultaneously achieve optimal performance, computational efficiency, and data privacy limits their application to LLMs. In this paper, we propose \textbf{M}odel \textbf{E}xclusive \textbf{T}ask \textbf{A}rithmetic for merging \textbf{GPT}-scale models, which formalizes the objective of model merging into a multi-task learning framework, aiming to minimize the average loss difference between the merged model and each individual task model. Since data privacy limits the use of multi-task training data, we leverage LLMs' local linearity and task vectors' orthogonality to separate the data term and scaling coefficients term and derive a model-exclusive task arithmetic method. Our proposed MetaGPT is data-agnostic and bypasses the heavy search process, making it cost-effective and easy to implement for LLMs.Extensive experiments demonstrate that MetaGPT leads to improvements in task arithmetic and achieves state-of-the-art performance on multiple tasks.
△ Less
Submitted 27 June, 2024; v1 submitted 17 June, 2024;
originally announced June 2024.