Showing 1–45 of 45 results for author: An, X

High-Fidelity Facial Albedo Estimation via Texture Quantization

Authors: Zimin Ran, Xingyu Ren, Xiang An, Kaicheng Yang, Xiangzi Dai, Ziyong Feng, Jia Guo, Linchao Zhu, Jiankang Deng

Abstract: Recent 3D face reconstruction methods have made significant progress in shape estimation, but high-fidelity facial albedo reconstruction remains challenging. Existing methods depend on expensive light-stage captured data to learn facial albedo maps. However, a lack of diversity in subjects limits their ability to recover high-fidelity results. In this paper, we present a novel facial albedo recons… ▽ More Recent 3D face reconstruction methods have made significant progress in shape estimation, but high-fidelity facial albedo reconstruction remains challenging. Existing methods depend on expensive light-stage captured data to learn facial albedo maps. However, a lack of diversity in subjects limits their ability to recover high-fidelity results. In this paper, we present a novel facial albedo reconstruction model, HiFiAlbedo, which recovers the albedo map directly from a single image without the need for captured albedo data. Our key insight is that the albedo map is the illumination invariant texture map, which enables us to use inexpensive texture data to derive an albedo estimation by eliminating illumination. To achieve this, we first collect large-scale ultra-high-resolution facial images and train a high-fidelity facial texture codebook. By using the FFHQ dataset and limited UV textures, we then fine-tune the encoder for texture reconstruction from the input image with adversarial supervision in both image and UV space. Finally, we train a cross-attention module and utilize group identity loss to learn the adaptation from facial texture to the albedo domain. Extensive experimentation has demonstrated that our method exhibits excellent generalizability and is capable of achieving high-fidelity results for in-the-wild facial albedo recovery. Our code, pre-trained weights, and training data will be made publicly available at https://hifialbedo.github.io/. △ Less

Submitted 18 June, 2024; originally announced June 2024.

RWKV-CLIP: A Robust Vision-Language Representation Learner

Authors: Tiancheng Gu, Kaicheng Yang, Xiang An, Ziyong Feng, Dongnan Liu, Weidong Cai, Jiankang Deng

Abstract: Contrastive Language-Image Pre-training (CLIP) has significantly improved performance in various vision-language tasks by expanding the dataset with image-text pairs obtained from websites. This paper further explores CLIP from the perspectives of data and model architecture. To address the prevalence of noisy data and enhance the quality of large-scale image-text data crawled from the internet, w… ▽ More Contrastive Language-Image Pre-training (CLIP) has significantly improved performance in various vision-language tasks by expanding the dataset with image-text pairs obtained from websites. This paper further explores CLIP from the perspectives of data and model architecture. To address the prevalence of noisy data and enhance the quality of large-scale image-text data crawled from the internet, we introduce a diverse description generation framework that can leverage Large Language Models (LLMs) to synthesize and refine content from web-based texts, synthetic captions, and detection tags. Furthermore, we propose RWKV-CLIP, the first RWKV-driven vision-language representation learning model that combines the effective parallel training of transformers with the efficient inference of RNNs. Comprehensive experiments across various model scales and pre-training datasets demonstrate that RWKV-CLIP is a robust and efficient vision-language representation learner, it achieves state-of-the-art performance in several downstream tasks, including linear probe, zero-shot classification, and zero-shot image-text retrieval. To facilitate future research, the code and pre-trained models are released at https://github.com/deepglint/RWKV-CLIP △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 14 pages, 10 figures

A Lightweight Method for Defending Against UAF Vulnerabilities

Authors: Xun An

Abstract: The widespread presence of Use-After-Free (UAF) vulnerabilities poses a serious threat to software security, with dangling pointers being considered the primary cause of these vulnerabilities. However, existing methods for defending against UAF vulnerabilities by eliminating dangling pointers need to interrupt the program's execution when encountering pointer assignment operations to look up the o… ▽ More The widespread presence of Use-After-Free (UAF) vulnerabilities poses a serious threat to software security, with dangling pointers being considered the primary cause of these vulnerabilities. However, existing methods for defending against UAF vulnerabilities by eliminating dangling pointers need to interrupt the program's execution when encountering pointer assignment operations to look up the objects pointed to by the pointers and store the memory addresses of the pointers in a specific data structure. This makes these methods not lightweight. To overcome this drawback, we propose a novel approach called LightDE. This method does not require storing the memory addresses of pointers or locating the objects pointed to by pointers during program execution. LightDE uses our proposed structure-sensitive pointer analysis method to determine the objects pointed to by pointers and stores the pointing relationships in the program's data segment during program compilation. Since LightDE only needs to check whether the pointers identified by the pointer analysis point to the released objects when the objects are released, LightDE is very lightweight. Our experimental results show that LightDE can effectively defend against UAF vulnerabilities, and the additional performance overhead it introduces is very low. △ Less

Submitted 31 May, 2024; originally announced May 2024.

Federated Learning with Only Positive Labels by Exploring Label Correlations

Authors: Xuming An, Dui Wang, Li Shen, Yong Luo, Han Hu, Bo Du, Yonggang Wen, Dacheng Tao

Abstract: Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting, where trivial solution and extremely poor performance may be obtained, especially when only positive data w.r.t. a single class label are provided for each client. This issue ca… ▽ More Federated learning aims to collaboratively learn a model by using the data from multiple users under privacy constraints. In this paper, we study the multi-label classification problem under the federated learning setting, where trivial solution and extremely poor performance may be obtained, especially when only positive data w.r.t. a single class label are provided for each client. This issue can be addressed by adding a specially designed regularizer on the server-side. Although effective sometimes, the label correlations are simply ignored and thus sub-optimal performance may be obtained. Besides, it is expensive and unsafe to exchange user's private embeddings between server and clients frequently, especially when training model in the contrastive way. To remedy these drawbacks, we propose a novel and generic method termed Federated Averaging by exploring Label Correlations (FedALC). Specifically, FedALC estimates the label correlations in the class embedding learning for different label pairs and utilizes it to improve the model training. To further improve the safety and also reduce the communication overhead, we propose a variant to learn fixed class embedding for each client, so that the server and clients only need to exchange class embeddings once. Extensive experiments on multiple popular datasets demonstrate that our FedALC can significantly outperform existing counterparts. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: To be published in IEEE Transactions on Neural Networks and Learning Systems

Domain adaptive pose estimation via multi-level alignment

Authors: Yugan Chen, Lin Zhao, Yalong Xu, Honglei Zu, Xiaoqi An, Guangyu Li

Abstract: Domain adaptive pose estimation aims to enable deep models trained on source domain (synthesized) datasets produce similar results on the target domain (real-world) datasets. The existing methods have made significant progress by conducting image-level or feature-level alignment. However, only aligning at a single level is not sufficient to fully bridge the domain gap and achieve excellent domain… ▽ More Domain adaptive pose estimation aims to enable deep models trained on source domain (synthesized) datasets produce similar results on the target domain (real-world) datasets. The existing methods have made significant progress by conducting image-level or feature-level alignment. However, only aligning at a single level is not sufficient to fully bridge the domain gap and achieve excellent domain adaptive results. In this paper, we propose a multi-level domain adaptation aproach, which aligns different domains at the image, feature, and pose levels. Specifically, we first utilize image style transer to ensure that images from the source and target domains have a similar distribution. Subsequently, at the feature level, we employ adversarial training to make the features from the source and target domains preserve domain-invariant characeristics as much as possible. Finally, at the pose level, a self-supervised approach is utilized to enable the model to learn diverse knowledge, implicitly addressing the domain gap. Experimental results demonstrate that significant imrovement can be achieved by the proposed multi-level alignment method in pose estimation, which outperforms previous state-of-the-art in human pose by up to 2.4% and animal pose estimation by up to 3.1% for dogs and 1.4% for sheep. △ Less

Submitted 25 April, 2024; v1 submitted 23 April, 2024; originally announced April 2024.

Comments: accepted to icme2024

Plug-and-Play Grounding of Reasoning in Multimodal Large Language Models

Authors: Jiaxing Chen, Yuxuan Liu, Dehu Li, Xiang An, Weimo Deng, Ziyong Feng, Yongle Zhao, Yin Xie

Abstract: The rise of Multimodal Large Language Models (MLLMs), renowned for their advanced instruction-following and reasoning capabilities, has significantly propelled the field of visual reasoning. However, due to limitations in their image tokenization processes, most MLLMs struggle to capture fine details of text and objects in images, especially in high-resolution samples. To overcome this limitation,… ▽ More The rise of Multimodal Large Language Models (MLLMs), renowned for their advanced instruction-following and reasoning capabilities, has significantly propelled the field of visual reasoning. However, due to limitations in their image tokenization processes, most MLLMs struggle to capture fine details of text and objects in images, especially in high-resolution samples. To overcome this limitation, we introduce P2G, a novel framework for plug-and-play grounding in MLLMs. P2G utilizes the tool-usage potential of MLLMs to employ expert agents for on-the-fly grounding of reasoning into critical visual and textual elements in images, thereby enabling deliberate reasoning through multimodal prompting. Additionally, we develop P2GB, a benchmark designed to evaluate MLLMs' proficiency in understanding inter-object relationships and textual content in challenging high-resolution images. Extensive experiments on visual reasoning tasks demonstrate the superiority of P2G, achieving performance comparable to GPT-4V on P2GB with a 7B backbone. Our work underscores the potential of grounding reasoning with external agents in MLLMs, presenting a promising alternative to mere model scaling. △ Less

Submitted 18 June, 2024; v1 submitted 28 March, 2024; originally announced March 2024.

Comments: 15 pages, 8 figures

IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models

Authors: Siying Cui, Jia Guo, Xiang An, Jiankang Deng, Yongle Zhao, Xinyu Wei, Ziyong Feng

Abstract: Leveraging Stable Diffusion for the generation of personalized portraits has emerged as a powerful and noteworthy tool, enabling users to create high-fidelity, custom character avatars based on their specific prompts. However, existing personalization methods face challenges, including test-time fine-tuning, the requirement of multiple input images, low preservation of identity, and limited divers… ▽ More Leveraging Stable Diffusion for the generation of personalized portraits has emerged as a powerful and noteworthy tool, enabling users to create high-fidelity, custom character avatars based on their specific prompts. However, existing personalization methods face challenges, including test-time fine-tuning, the requirement of multiple input images, low preservation of identity, and limited diversity in generated outcomes. To overcome these challenges, we introduce IDAdapter, a tuning-free approach that enhances the diversity and identity preservation in personalized image generation from a single face image. IDAdapter integrates a personalized concept into the generation process through a combination of textual and visual injections and a face identity loss. During the training phase, we incorporate mixed features from multiple reference images of a specific identity to enrich identity-related content details, guiding the model to generate images with more diverse styles, expressions, and angles compared to previous works. Extensive evaluations demonstrate the effectiveness of our method, achieving both diversity and identity fidelity in generated images. △ Less

Submitted 20 March, 2024; v1 submitted 20 March, 2024; originally announced March 2024.

Comments: 14 pages, 15 figures

Enabling Communication and Control Co-Design in 6G Networks

Authors: Onur Ayan, Nikolaos Pappas, Miguel Angel Gutierrez Estevez, Xueli An, Wolfgang Kellerer

Abstract: Networked control systems (NCSs), which are feedback control loops closed over a communication network, have been a popular research topic over the past decades. Numerous works in the literature propose novel algorithms and protocols with joint consideration of communication and control. However, the vast majority of the recent research results, which have shown remarkable performance improvements… ▽ More Networked control systems (NCSs), which are feedback control loops closed over a communication network, have been a popular research topic over the past decades. Numerous works in the literature propose novel algorithms and protocols with joint consideration of communication and control. However, the vast majority of the recent research results, which have shown remarkable performance improvements if a cross-layer methodology is followed, have not been widely adopted by the industry. In this work, we review the shortcomings of today's mobile networks that render cross-layer solutions, such as semantic and goal-oriented communications, very challenging in practice. To tackle this, we propose a new framework for 6G user plane design that simplifies the adoption of recent research results in networked control, thereby facilitating the joint communication and control design in next-generation mobile networks. △ Less

Submitted 26 February, 2024; originally announced February 2024.

SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation

Authors: Xiaoqi An, Lin Zhao, Chen Gong, Nannan Wang, Di Wang, Jian Yang

Abstract: High-resolution representation is essential for achieving good performance in human pose estimation models. To obtain such features, existing works utilize high-resolution input images or fine-grained image tokens. However, this dense high-resolution representation brings a significant computational burden. In this paper, we address the following question: "Only sparse human keypoint locations are… ▽ More High-resolution representation is essential for achieving good performance in human pose estimation models. To obtain such features, existing works utilize high-resolution input images or fine-grained image tokens. However, this dense high-resolution representation brings a significant computational burden. In this paper, we address the following question: "Only sparse human keypoint locations are detected for human pose estimation, is it really necessary to describe the whole image in a dense, high-resolution manner?" Based on dynamic transformer models, we propose a framework that only uses Sparse High-resolution Representations for human Pose estimation (SHaRPose). In detail, SHaRPose consists of two stages. At the coarse stage, the relations between image regions and keypoints are dynamically mined while a coarse estimation is generated. Then, a quality predictor is applied to decide whether the coarse estimation results should be refined. At the fine stage, SHaRPose builds sparse high-resolution representations only on the regions related to the keypoints and provides refined high-precision human pose estimations. Extensive experiments demonstrate the outstanding performance of the proposed method. Specifically, compared to the state-of-the-art method ViTPose, our model SHaRPose-Base achieves 77.4 AP (+0.5 AP) on the COCO validation set and 76.7 AP (+0.5 AP) on the COCO test-dev set, and infers at a speed of $1.4\times$ faster than ViTPose-Base. △ Less

Submitted 17 December, 2023; originally announced December 2023.

Comments: Accepted to AAAI 2024

The High-dimensional Phase Diagram and the Large CALPHAD Model

Authors: Zhengdi Liu, Xulong An, Wenwen Sun

Abstract: When alloy systems comprise more than three elements, the visualization of the entire phase space becomes not only daunting but is also accompanied by a data surge. Addressing this complexity, we delve into the FeNiCrMn alloy system and introduce the Large CALPHAD Model (LCM). The LCM acts as a computational conduit, capturing the entire phase space. Subsequently, this enormous data is systematica… ▽ More When alloy systems comprise more than three elements, the visualization of the entire phase space becomes not only daunting but is also accompanied by a data surge. Addressing this complexity, we delve into the FeNiCrMn alloy system and introduce the Large CALPHAD Model (LCM). The LCM acts as a computational conduit, capturing the entire phase space. Subsequently, this enormous data is systematically structured using a high-dimensional phase diagram, aided by hash tables and Depth-first Search (DFS), rendering it both digestible and programmatically accessible. Remarkably, the LCM boasts a 97% classification accuracy and a mean square error of 4.80*10-5 in phase volume prediction. Our methodology successfully delineates 51 unique phase spaces in the FeNiCrMn system, exemplifying its efficacy with the design of all 439 eutectic alloys. This pioneering methodology signifies a monumental shift in alloy design techniques or even multi-variable problems. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Federated Learning with Manifold Regularization and Normalized Update Reaggregation

Authors: Xuming An, Li Shen, Han Hu, Yong Luo

Abstract: Federated Learning (FL) is an emerging collaborative machine learning framework where multiple clients train the global model without sharing their own datasets. In FL, the model inconsistency caused by the local data heterogeneity across clients results in the near-orthogonality of client updates, which leads to the global update norm reduction and slows down the convergence. Most previous works… ▽ More Federated Learning (FL) is an emerging collaborative machine learning framework where multiple clients train the global model without sharing their own datasets. In FL, the model inconsistency caused by the local data heterogeneity across clients results in the near-orthogonality of client updates, which leads to the global update norm reduction and slows down the convergence. Most previous works focus on eliminating the difference of parameters (or gradients) between the local and global models, which may fail to reflect the model inconsistency due to the complex structure of the machine learning model and the Euclidean space's limitation in meaningful geometric representations. In this paper, we propose FedMRUR by adopting the manifold model fusion scheme and a new global optimizer to alleviate the negative impacts. Concretely, FedMRUR adopts a hyperbolic graph manifold regularizer enforcing the representations of the data in the local and global models are close to each other in a low-dimensional subspace. Because the machine learning model has the graph structure, the distance in hyperbolic space can reflect the model bias better than the Euclidean distance. In this way, FedMRUR exploits the manifold structures of the representations to significantly reduce the model inconsistency. FedMRUR also aggregates the client updates norms as the global update norm, which can appropriately enlarge each client's contribution to the global update, thereby mitigating the norm reduction introduced by the near-orthogonality of client updates. Furthermore, we theoretically prove that our algorithm can achieve a linear speedup property for non-convex setting under partial client participation.Experiments demonstrate that FedMRUR can achieve a new state-of-the-art (SOTA) accuracy with less communication. △ Less

Submitted 10 November, 2023; originally announced November 2023.

Multi-structure Objects Points-to Analysis

Authors: Xun An

Abstract: An important dimension of pointer analysis is field-Sensitive, which has been proven to effectively enhance the accuracy of pointer analysis results. A crucial area of research within field-Sensitive is Structure-Sensitive. Structure-Sensitive has been shown to further enhance the precision of pointer analysis. However, existing structure-sensitive methods cannot handle cases where an object posse… ▽ More An important dimension of pointer analysis is field-Sensitive, which has been proven to effectively enhance the accuracy of pointer analysis results. A crucial area of research within field-Sensitive is Structure-Sensitive. Structure-Sensitive has been shown to further enhance the precision of pointer analysis. However, existing structure-sensitive methods cannot handle cases where an object possesses multiple structures, even though it's common for an object to have multiple structures throughout its lifecycle. This paper introduces MTO-SS, a flow-sensitive pointer analysis method for objects with multiple structures. Our observation is that it's common for an object to possess multiple structures throughout its lifecycle. The novelty of MTO-SS lies in: MTO-SS introduces Structure-Flow-Sensitive. An object has different structure information at different locations in the program. To ensure the completeness of an object's structure information, MTO-SS always performs weak updates on the object's type. This means that once an object possesses a structure, this structure will accompany the object throughout its lifecycle. We evaluated our method of multi-structured object pointer analysis using the 12 largest programs in GNU Coreutils and compared the experimental results with sparse flow-sensitive method and another method, TYPECLONE, which only allows an object to have one structure information. Our experimental results confirm that MTO-SS is more precise than both sparse flow-sensitive pointer analysis and TYPECLONE, being able to answer, on average, over 22\% more alias queries with a no-alias result compared to the former, and over 3\% more compared to the latter. Additionally, the time overhead introduced by our method is very low. △ Less

Submitted 6 November, 2023; v1 submitted 25 October, 2023; originally announced October 2023.

SGORP: A Subgradient-based Method for d-Dimensional Rectilinear Partitioning

Authors: Muhammed Fatih Balin, Xiao**g An, Abdurrahman Yaşar, Ümit V. Çatalyürek

Abstract: Partitioning for load balancing is a crucial first step to parallelize any type of computation. In this work, we propose SGORP, a new spatial partitioning method based on Subgradient Optimization, to solve the $d$-dimensional Rectilinear Partitioning Problem (RPP). Our proposed method allows the use of customizable objective functions as well as some user-specific constraints, such as symmetric pa… ▽ More Partitioning for load balancing is a crucial first step to parallelize any type of computation. In this work, we propose SGORP, a new spatial partitioning method based on Subgradient Optimization, to solve the $d$-dimensional Rectilinear Partitioning Problem (RPP). Our proposed method allows the use of customizable objective functions as well as some user-specific constraints, such as symmetric partitioning on selected dimensions. Extensive experimental evaluation using over 600 test matrices shows that our algorithm achieves favorable performance against the state-of-the-art RPP and Symmetric RPP algorithms. Additionally, we show the effectiveness of our algorithm to do application-specific load balancing using two applications as motivation: Triangle Counting and Sparse Matrix Multiplication (SpGEMM), where we model their load-balancing problems as $3$-dimensional RPPs. △ Less

Submitted 3 October, 2023; originally announced October 2023.

Integrated Sensing and Communications for IoT: Synergies with Key 6G Technology Enablers

Authors: Aryan Kaushik, Rohit Singh, Ming Li, Honghao Luo, Shalanika Dayarathna, Rajitha Senanayake, Xueli An, Richard A. Stirling-Gallacher, Wonjae Shin, Marco Di Renzo

Abstract: The Internet of Things (IoT) and wireless generations have been evolving simultaneously for the past few decades. Built upon wireless communication and sensing technologies, IoT networks are usually evaluated based on metrics that measure the device ability to sense information and effectively share it with the network, which makes Integrated Sensing and Communication (ISAC) a pivotal candidate fo… ▽ More The Internet of Things (IoT) and wireless generations have been evolving simultaneously for the past few decades. Built upon wireless communication and sensing technologies, IoT networks are usually evaluated based on metrics that measure the device ability to sense information and effectively share it with the network, which makes Integrated Sensing and Communication (ISAC) a pivotal candidate for the sixth-generation (6G) IoT standards. This paper reveals several innovative aspects of ISAC from an IoT perspective in 6G, empowering various modern IoT use cases and key technology enablers. Moreover, we address the challenges and future potential of ISAC-enabled IoT, including synergies with Reconfigurable Intelligent Surfaces (RIS), Artificial Intelligence (AI), and key updates of ISAC-IoT in 6G standardization. Furthermore, several evolutionary concepts are introduced to open future research in 6G ISAC-IoT, including the interplay with Non-Terrestrial Networks (NTN) and Orthogonal Time-Frequency Space (OTFS) modulation. △ Less

Submitted 23 September, 2023; originally announced September 2023.

Comments: 7 pages, 6 figures

Advancing Federated Learning in 6G: A Trusted Architecture with Graph-based Analysis

Authors: Wenxuan Ye, Chendi Qian, Xueli An, Xueqiang Yan, Georg Carle

Abstract: Integrating native AI support into the network architecture is an essential objective of 6G. Federated Learning (FL) emerges as a potential paradigm, facilitating decentralized AI model training across a diverse range of devices under the coordination of a central server. However, several challenges hinder its wide application in the 6G context, such as malicious attacks and privacy snoo** on lo… ▽ More Integrating native AI support into the network architecture is an essential objective of 6G. Federated Learning (FL) emerges as a potential paradigm, facilitating decentralized AI model training across a diverse range of devices under the coordination of a central server. However, several challenges hinder its wide application in the 6G context, such as malicious attacks and privacy snoo** on local model updates, and centralization pitfalls. This work proposes a trusted architecture for supporting FL, which utilizes Distributed Ledger Technology (DLT) and Graph Neural Network (GNN), including three key features. First, a pre-processing layer employing homomorphic encryption is incorporated to securely aggregate local models, preserving the privacy of individual models. Second, given the distributed nature and graph structure between clients and nodes in the pre-processing layer, GNN is leveraged to identify abnormal local models, enhancing system security. Third, DLT is utilized to decentralize the system by selecting one of the candidates to perform the central server's functions. Additionally, DLT ensures reliable data management by recording data exchanges in an immutable and transparent ledger. The feasibility of the novel architecture is validated through simulations, demonstrating improved performance in anomalous model detection and global model accuracy compared to relevant baselines. △ Less

Submitted 27 September, 2023; v1 submitted 11 September, 2023; originally announced September 2023.

Comments: Accepted by IEEE Global Communications Conference (GLOBECOM) 2023

Asymmetric double-winged multi-view clustering network for exploring Diverse and Consistent Information

Authors: Qun Zheng, Xihong Yang, Siwei Wang, Xinru An, Qi Liu

Abstract: In unsupervised scenarios, deep contrastive multi-view clustering (DCMVC) is becoming a hot research spot, which aims to mine the potential relationships between different views. Most existing DCMVC algorithms focus on exploring the consistency information for the deep semantic features, while ignoring the diverse information on shallow features. To fill this gap, we propose a novel multi-view clu… ▽ More In unsupervised scenarios, deep contrastive multi-view clustering (DCMVC) is becoming a hot research spot, which aims to mine the potential relationships between different views. Most existing DCMVC algorithms focus on exploring the consistency information for the deep semantic features, while ignoring the diverse information on shallow features. To fill this gap, we propose a novel multi-view clustering network termed CodingNet to explore the diverse and consistent information simultaneously in this paper. Specifically, instead of utilizing the conventional auto-encoder, we design an asymmetric structure network to extract shallow and deep features separately. Then, by aligning the similarity matrix on the shallow feature to the zero matrix, we ensure the diversity for the shallow features, thus offering a better description of multi-view data. Moreover, we propose a dual contrastive mechanism that maintains consistency for deep features at both view-feature and pseudo-label levels. Our framework's efficacy is validated through extensive experiments on six widely used benchmark datasets, outperforming most state-of-the-art multi-view clustering algorithms. △ Less

Submitted 1 September, 2023; originally announced September 2023.

Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient

Authors: Rongfei Fan, Xuming An, Shiyuan Zuo, Han Hu

Abstract: Over-the-air computation is a communication-efficient solution for federated learning (FL). In such a system, iterative procedure is performed: Local gradient of private loss function is updated, amplified and then transmitted by every mobile device; the server receives the aggregated gradient all-at-once, generates and then broadcasts updated model parameters to every mobile device. In terms of a… ▽ More Over-the-air computation is a communication-efficient solution for federated learning (FL). In such a system, iterative procedure is performed: Local gradient of private loss function is updated, amplified and then transmitted by every mobile device; the server receives the aggregated gradient all-at-once, generates and then broadcasts updated model parameters to every mobile device. In terms of amplification factor selection, most related works suppose the local gradient's maximal norm always happens although it actually fluctuates over iterations, which may degrade convergence performance. To circumvent this problem, we propose to turn local gradient to be normalized one before amplifying it. Under our proposed method, when the loss function is smooth, we prove our proposed method can converge to stationary point at sub-linear rate. In case of smooth and strongly convex loss function, we prove our proposed method can achieve minimal training loss at linear rate with any small positive tolerance. Moreover, a tradeoff between convergence rate and the tolerance is discovered. To speedup convergence, problems optimizing system parameters are also formulated for above two cases. Although being non-convex, optimal solution with polynomial complexity of the formulated problems are derived. Experimental results show our proposed method can outperform benchmark methods on convergence performance. △ Less

Submitted 2 September, 2023; v1 submitted 17 August, 2023; originally announced August 2023.

Joint Power Control and Data Size Selection for Over-the-Air Computation Aided Federated Learning

Authors: Xuming An, Rongfei Fan, Shiyuan Zuo, Han Hu, Hai Jiang, Ning Zhang

Abstract: Federated learning (FL) has emerged as an appealing machine learning approach to deal with massive raw data generated at multiple mobile devices, {which needs to aggregate the training model parameter of every mobile device at one base station (BS) iteratively}. For parameter aggregating in FL, over-the-air computation is a spectrum-efficient solution, which allows all mobile devices to transmit t… ▽ More Federated learning (FL) has emerged as an appealing machine learning approach to deal with massive raw data generated at multiple mobile devices, {which needs to aggregate the training model parameter of every mobile device at one base station (BS) iteratively}. For parameter aggregating in FL, over-the-air computation is a spectrum-efficient solution, which allows all mobile devices to transmit their parameter-mapped signals concurrently to a BS. Due to heterogeneous channel fading and noise, there exists difference between the BS's received signal and its desired signal, measured as the mean-squared error (MSE). To minimize the MSE, we propose to jointly optimize the signal amplification factors at the BS and the mobile devices as well as the data size (the number of data samples involved in local training) at every mobile device. The formulated problem is challenging to solve due to its non-convexity. To find the optimal solution, with some simplification on cost function and variable replacement, which still preserves equivalence, we transform the changed problem to be a bi-level problem equivalently. For the lower-level problem, optimal solution is found by enumerating every candidate solution from the Karush-Kuhn-Tucker (KKT) condition. For the upper-level problem, the optimal solution is found by exploring its piecewise convexity. Numerical results show that our proposed method can greatly reduce the MSE and can help to improve the training performance of FL compared with benchmark methods. △ Less

Submitted 17 August, 2023; originally announced August 2023.

ALIP: Adaptive Language-Image Pre-training with Synthetic Caption

Authors: Kaicheng Yang, Jiankang Deng, Xiang An, Jiawei Li, Ziyong Feng, Jia Guo, **g Yang, Tongliang Liu

Abstract: Contrastive Language-Image Pre-training (CLIP) has significantly boosted the performance of various vision-language tasks by scaling up the dataset with image-text pairs collected from the web. However, the presence of intrinsic noise and unmatched image-text pairs in web data can potentially affect the performance of representation learning. To address this issue, we first utilize the OFA model t… ▽ More Contrastive Language-Image Pre-training (CLIP) has significantly boosted the performance of various vision-language tasks by scaling up the dataset with image-text pairs collected from the web. However, the presence of intrinsic noise and unmatched image-text pairs in web data can potentially affect the performance of representation learning. To address this issue, we first utilize the OFA model to generate synthetic captions that focus on the image content. The generated captions contain complementary information that is beneficial for pre-training. Then, we propose an Adaptive Language-Image Pre-training (ALIP), a bi-path model that integrates supervision from both raw text and synthetic caption. As the core components of ALIP, the Language Consistency Gate (LCG) and Description Consistency Gate (DCG) dynamically adjust the weights of samples and image-text/caption pairs during the training process. Meanwhile, the adaptive contrastive loss can effectively reduce the impact of noise data and enhances the efficiency of pre-training data. We validate ALIP with experiments on different scales of models and pre-training datasets. Experiments results show that ALIP achieves state-of-the-art performance on multiple downstream tasks including zero-shot image-text retrieval and linear probe. To facilitate future research, the code and pre-trained models are released at https://github.com/deepglint/ALIP. △ Less

Submitted 18 August, 2023; v1 submitted 16 August, 2023; originally announced August 2023.

Comments: 15pages, 10figures, ICCV2023

Distributed Sensing, Computing, Communication, and Control Fabric: A Unified Service-Level Architecture for 6G

Authors: Dejan Vukobratović, Nikolaos Bartzoudis, Mona Ghassemian, Firooz Saghezchi, Peizheng Li, Adnan Aijaz, Ricardo Martinez, Xueli An, Ranga Rao Venkatesha Prasad, Helge Lüders, Shahid Mumtaz

Abstract: With the advent of the multimodal immersive communication system, people can interact with each other using multiple devices for sensing, communication and/or control either onsite or remotely. As a breakthrough concept, a distributed sensing, computing, communications, and control (DS3C) fabric is introduced in this paper for provisioning 6G services in multi-tenant environments in a unified mann… ▽ More With the advent of the multimodal immersive communication system, people can interact with each other using multiple devices for sensing, communication and/or control either onsite or remotely. As a breakthrough concept, a distributed sensing, computing, communications, and control (DS3C) fabric is introduced in this paper for provisioning 6G services in multi-tenant environments in a unified manner. The DS3C fabric can be further enhanced by natively incorporating intelligent algorithms for network automation and managing networking, computing, and sensing resources efficiently to serve vertical use cases with extreme and/or conflicting requirements. As such, the paper proposes a novel end-to-end 6G system architecture with enhanced intelligence spanning across different network, computing, and business domains, identifies vertical use cases and presents an overview of the relevant standardization and pre-standardization landscape. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Unicom: Universal and Compact Representation Learning for Image Retrieval

Authors: Xiang An, Jiankang Deng, Kaicheng Yang, Jaiwei Li, Ziyong Feng, Jia Guo, **g Yang, Tongliang Liu

Abstract: Modern image retrieval methods typically rely on fine-tuning pre-trained encoders to extract image-level descriptors. However, the most widely used models are pre-trained on ImageNet-1K with limited classes. The pre-trained feature representation is therefore not universal enough to generalize well to the diverse open-world classes. In this paper, we first cluster the large-scale LAION400M into on… ▽ More Modern image retrieval methods typically rely on fine-tuning pre-trained encoders to extract image-level descriptors. However, the most widely used models are pre-trained on ImageNet-1K with limited classes. The pre-trained feature representation is therefore not universal enough to generalize well to the diverse open-world classes. In this paper, we first cluster the large-scale LAION400M into one million pseudo classes based on the joint textual and visual features extracted by the CLIP model. Due to the confusion of label granularity, the automatically clustered dataset inevitably contains heavy inter-class conflict. To alleviate such conflict, we randomly select partial inter-class prototypes to construct the margin-based softmax loss. To further enhance the low-dimensional feature representation, we randomly select partial feature dimensions when calculating the similarities between embeddings and class-wise prototypes. The dual random partial selections are with respect to the class dimension and the feature dimension of the prototype matrix, making the classification conflict-robust and the feature embedding compact. Our method significantly outperforms state-of-the-art unsupervised and supervised image retrieval approaches on multiple benchmarks. The code and pre-trained models are released to facilitate future research https://github.com/deepglint/unicom. △ Less

Submitted 12 April, 2023; originally announced April 2023.

Comments: Accepted at ICLR2023

Quadratic Graph Attention Network (Q-GAT) for Robust Construction of Gene Regulatory Networks

Authors: Hui Zhang, Xuexin An, Qiang He, Yudong Yao, Yudong Zhang, Feng-Lei Fan, Yueyang Teng

Abstract: Gene regulatory relationships can be abstracted as a gene regulatory network (GRN), which plays a key role in characterizing complex cellular processes and pathways. Recently, graph neural networks (GNNs), as a class of deep learning models, have emerged as a useful tool to infer gene regulatory relationships from gene expression data. However, deep learning models have been found to be vulnerable… ▽ More Gene regulatory relationships can be abstracted as a gene regulatory network (GRN), which plays a key role in characterizing complex cellular processes and pathways. Recently, graph neural networks (GNNs), as a class of deep learning models, have emerged as a useful tool to infer gene regulatory relationships from gene expression data. However, deep learning models have been found to be vulnerable to noise, which greatly hinders the adoption of deep learning in constructing GRNs, because high noise is often unavoidable in the process of gene expression measurement. Can we preferably prototype a robust GNN for constructing GRNs? In this paper, we give a positive answer by proposing a Quadratic Graph Attention Network (Q-GAT) with a dual attention mechanism. We study the changes in the predictive accuracy of Q-GAT and 9 state-of-the-art baselines by introducing different levels of adversarial perturbations. Experiments in the E. coli and S. cerevisiae datasets suggest that Q-GAT outperforms the state-of-the-art models in robustness. Lastly, we dissect why Q-GAT is robust through the signal-to-noise ratio (SNR) and interpretability analyses. The former informs that nonlinear aggregation of quadratic neurons can amplify useful signals and suppress unwanted noise, thereby facilitating robustness, while the latter reveals that Q-GAT can leverage more features in prediction thanks to the dual attention mechanism, which endows Q-GAT with the ability to confront adversarial perturbation. We have shared our code in https://github.com/Minorway/Q-GAT_for_Robust_Construction_of_GRN for readers' evaluation. △ Less

Submitted 4 November, 2023; v1 submitted 23 March, 2023; originally announced March 2023.

Uncertainty-Aware Reward-based Deep Reinforcement Learning for Intent Analysis of Social Media Information

Authors: Zhen Guo, Qi Zhang, Xinwei An, Qisheng Zhang, Audun Jøsang, Lance M. Kaplan, Feng Chen, Dong H. Jeong, **-Hee Cho

Abstract: Due to various and serious adverse impacts of spreading fake news, it is often known that only people with malicious intent would propagate fake news. However, it is not necessarily true based on social science studies. Distinguishing the types of fake news spreaders based on their intent is critical because it will effectively guide how to intervene to mitigate the spread of fake news with differ… ▽ More Due to various and serious adverse impacts of spreading fake news, it is often known that only people with malicious intent would propagate fake news. However, it is not necessarily true based on social science studies. Distinguishing the types of fake news spreaders based on their intent is critical because it will effectively guide how to intervene to mitigate the spread of fake news with different approaches. To this end, we propose an intent classification framework that can best identify the correct intent of fake news. We will leverage deep reinforcement learning (DRL) that can optimize the structural representation of each tweet by removing noisy words from the input sequence when appending an actor to the long short-term memory (LSTM) intent classifier. Policy gradient DRL model (e.g., REINFORCE) can lead the actor to a higher delayed reward. We also devise a new uncertainty-aware immediate reward using a subjective opinion that can explicitly deal with multidimensional uncertainty for effective decision-making. Via 600K training episodes from a fake news tweets dataset with an annotated intent class, we evaluate the performance of uncertainty-aware reward in DRL. Evaluation results demonstrate that our proposed framework efficiently reduces the number of selected words to maintain a high 95\% multi-class accuracy. △ Less

Submitted 18 February, 2023; originally announced February 2023.

Comments: Accepted version

On Heterogeneous Treatment Effects in Heterogeneous Causal Graphs

Authors: Richard A Watson, Hengrui Cai, Xinming An, Samuel McLean, Rui Song

Abstract: Heterogeneity and comorbidity are two interwoven challenges associated with various healthcare problems that greatly hampered research on develo** effective treatment and understanding of the underlying neurobiological mechanism. Very few studies have been conducted to investigate heterogeneous causal effects (HCEs) in graphical contexts due to the lack of statistical methods. To characterize th… ▽ More Heterogeneity and comorbidity are two interwoven challenges associated with various healthcare problems that greatly hampered research on develo** effective treatment and understanding of the underlying neurobiological mechanism. Very few studies have been conducted to investigate heterogeneous causal effects (HCEs) in graphical contexts due to the lack of statistical methods. To characterize this heterogeneity, we first conceptualize heterogeneous causal graphs (HCGs) by generalizing the causal graphical model with confounder-based interactions and multiple mediators. Such confounders with an interaction with the treatment are known as moderators. This allows us to flexibly produce HCGs given different moderators and explicitly characterize HCEs from the treatment or potential mediators on the outcome. We establish the theoretical forms of HCEs and derive their properties at the individual level in both linear and nonlinear models. An interactive structural learning is developed to estimate the complex HCGs and HCEs with confidence intervals provided. Our method is empirically justified by extensive simulations and its practical usefulness is illustrated by exploring causality among psychiatric disorders for trauma survivors. △ Less

Submitted 25 June, 2023; v1 submitted 29 January, 2023; originally announced January 2023.

Comments: In Proceedings of the 40th International Conference on Machine Learning (ICML) Code implementing the proposed algorithm is open-source and publicly available at: https://github.com/richard-watson/ISL

Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FC

Authors: Xiang An, Jiankang Deng, Jia Guo, Ziyong Feng, Xuhan Zhu, **g Yang, Tongliang Liu

Abstract: Learning discriminative deep feature embeddings by using million-scale in-the-wild datasets and margin-based softmax loss is the current state-of-the-art approach for face recognition. However, the memory and computing cost of the Fully Connected (FC) layer linearly scales up to the number of identities in the training set. Besides, the large-scale training data inevitably suffers from inter-class… ▽ More Learning discriminative deep feature embeddings by using million-scale in-the-wild datasets and margin-based softmax loss is the current state-of-the-art approach for face recognition. However, the memory and computing cost of the Fully Connected (FC) layer linearly scales up to the number of identities in the training set. Besides, the large-scale training data inevitably suffers from inter-class conflict and long-tailed distribution. In this paper, we propose a sparsely updating variant of the FC layer, named Partial FC (PFC). In each iteration, positive class centers and a random subset of negative class centers are selected to compute the margin-based softmax loss. All class centers are still maintained throughout the whole training process, but only a subset is selected and updated in each iteration. Therefore, the computing requirement, the probability of inter-class conflict, and the frequency of passive update on tail class centers, are dramatically reduced. Extensive experiments across different training data and backbones (e.g. CNN and ViT) confirm the effectiveness, robustness and efficiency of the proposed PFC. The source code is available at \https://github.com/deepinsight/insightface/tree/master/recognition. △ Less

Submitted 28 March, 2022; originally announced March 2022.

Comments: 8 pages, 16 figures

Journal ref: CVPR2022

Disentangling Style and Speaker Attributes for TTS Style Transfer

Authors: Xiaochun An, Frank K. Soong, Lei Xie

Abstract: End-to-end neural TTS has shown improved performance in speech style transfer. However, the improvement is still limited by the available training data in both target styles and speakers. Additionally, degenerated performance is observed when the trained TTS tries to transfer the speech to a target style from a new speaker with an unknown, arbitrary style. In this paper, we propose a new approach… ▽ More End-to-end neural TTS has shown improved performance in speech style transfer. However, the improvement is still limited by the available training data in both target styles and speakers. Additionally, degenerated performance is observed when the trained TTS tries to transfer the speech to a target style from a new speaker with an unknown, arbitrary style. In this paper, we propose a new approach to seen and unseen style transfer training on disjoint, multi-style datasets, i.e., datasets of different styles are recorded, one individual style by one speaker in multiple utterances. An inverse autoregressive flow (IAF) technique is first introduced to improve the variational inference for learning an expressive style representation. A speaker encoder network is then developed for learning a discriminative speaker embedding, which is jointly trained with the rest neural TTS modules. The proposed approach of seen and unseen style transfer is effectively trained with six specifically-designed objectives: reconstruction loss, adversarial loss, style distortion loss, cycle consistency loss, style classification loss, and speaker classification loss. Experiments demonstrate, both objectively and subjectively, the effectiveness of the proposed approach for seen and unseen style transfer tasks. The performance of our approach is superior to and more robust than those of four other reference systems of prior art. △ Less

Submitted 24 January, 2022; originally announced January 2022.

Masked Face Recognition Challenge: The InsightFace Track Report

Authors: Jiankang Deng, Jia Guo, Xiang An, Zheng Zhu, Stefanos Zafeiriou

Abstract: During the COVID-19 coronavirus epidemic, almost everyone wears a facial mask, which poses a huge challenge to deep face recognition. In this workshop, we organize Masked Face Recognition (MFR) challenge and focus on bench-marking deep face recognition methods under the existence of facial masks. In the MFR challenge, there are two main tracks: the InsightFace track and the WebFace260M track. For… ▽ More During the COVID-19 coronavirus epidemic, almost everyone wears a facial mask, which poses a huge challenge to deep face recognition. In this workshop, we organize Masked Face Recognition (MFR) challenge and focus on bench-marking deep face recognition methods under the existence of facial masks. In the MFR challenge, there are two main tracks: the InsightFace track and the WebFace260M track. For the InsightFace track, we manually collect a large-scale masked face test set with 7K identities. In addition, we also collect a children test set including 14K identities and a multi-racial test set containing 242K identities. By using these three test sets, we build up an online model testing system, which can give a comprehensive evaluation of face recognition models. To avoid data privacy problems, no test image is released to the public. As the challenge is still under-going, we will keep on updating the top-ranked solutions as well as this report on the arxiv. △ Less

Submitted 18 August, 2021; originally announced August 2021.

Comments: The WebFace260M Track of ICCV-21 MFR Challenge is still open in https://github.com/deepinsight/insightface/tree/master/challenges/iccv21-mfr

Improving Performance of Seen and Unseen Speech Style Transfer in End-to-end Neural TTS

Authors: Xiaochun An, Frank K. Soong, Lei Xie

Abstract: End-to-end neural TTS training has shown improved performance in speech style transfer. However, the improvement is still limited by the training data in both target styles and speakers. Inadequate style transfer performance occurs when the trained TTS tries to transfer the speech to a target style from a new speaker with an unknown, arbitrary style. In this paper, we propose a new approach to sty… ▽ More End-to-end neural TTS training has shown improved performance in speech style transfer. However, the improvement is still limited by the training data in both target styles and speakers. Inadequate style transfer performance occurs when the trained TTS tries to transfer the speech to a target style from a new speaker with an unknown, arbitrary style. In this paper, we propose a new approach to style transfer for both seen and unseen styles, with disjoint, multi-style datasets, i.e., datasets of different styles are recorded, each individual style is by one speaker with multiple utterances. To encode the style information, we adopt an inverse autoregressive flow (IAF) structure to improve the variational inference. The whole system is optimized to minimize a weighed sum of four different loss functions: 1) a reconstruction loss to measure the distortions in both source and target reconstructions; 2) an adversarial loss to "fool" a well-trained discriminator; 3) a style distortion loss to measure the expected style loss after the transfer; 4) a cycle consistency loss to preserve the speaker identity of the source after the transfer. Experiments demonstrate, both objectively and subjectively, the effectiveness of the proposed approach for seen and unseen style transfer tasks. The performance of the new approach is better and more robust than those of four baseline systems of the prior art. △ Less

Submitted 18 June, 2021; originally announced June 2021.

Toward Native Artificial Intelligence in 6G Networks: System Design, Architectures, and Paradigms

Authors: Jianjun Wu, Rongpeng Li, Xueli An, Chenghui Peng, Zhe Liu, Jon Crowcroft, Honggang Zhang

Abstract: The mobile communication system has transformed to be the fundamental infrastructure to support digital demands from all industry sectors, and 6G is envisioned to go far beyond the communication-only purpose. There is coming to a consensus that 6G will treat Artificial Intelligence (AI) as the cornerstone and has a potential capability to provide "intelligence inclusion", which implies to enable t… ▽ More The mobile communication system has transformed to be the fundamental infrastructure to support digital demands from all industry sectors, and 6G is envisioned to go far beyond the communication-only purpose. There is coming to a consensus that 6G will treat Artificial Intelligence (AI) as the cornerstone and has a potential capability to provide "intelligence inclusion", which implies to enable the access of AI services at anytime and anywhere by anyone. Apparently, the intelligent inclusion vision produces far-reaching influence on the corresponding network architecture design in 6G and deserves a clean-slate rethink. In this article, we propose an end-to-end system architecture design scope for 6G, and talk about the necessity to incorporate an independent data plane and a novel intelligent plane with particular emphasis on end-to-end AI workflow orchestration, management and operation. We also highlight the advantages to provision converged connectivity and computing services at the network function plane. Benefiting from these approaches, we believe that 6G will turn to an "everything as a service" (XaaS) platform with significantly enhanced business merits. △ Less

Submitted 3 March, 2021; originally announced March 2021.

Joint Task Offloading and Resource Allocation for IoT Edge Computing with Sequential Task Dependency

Authors: Xuming An, Rongfei Fan, Han Hu, Ning Zhang, Saman Atapattu, Theodoros A. Tsiftsis

Abstract: Incorporating mobile edge computing (MEC) in Internet of Things (IoT) enables resource-limited IoT devices to offload their computation tasks to a nearby edge server. In this paper, we investigate an IoT system assisted by the MEC technique with its computation task subjected to sequential task dependency, which is critical for video stream processing and other intelligent applications. To minimiz… ▽ More Incorporating mobile edge computing (MEC) in Internet of Things (IoT) enables resource-limited IoT devices to offload their computation tasks to a nearby edge server. In this paper, we investigate an IoT system assisted by the MEC technique with its computation task subjected to sequential task dependency, which is critical for video stream processing and other intelligent applications. To minimize energy consumption per IoT device while limiting task processing delay, task offloading strategy, communication resource, and computation resource are optimized jointly under both slow and fast fading channels. In slow fading channels, an optimization problem is formulated, which is non-convex and involves one integer variable. To solve this challenging problem, we decompose it as a one-dimensional search of task offloading decision problem and a non-convex optimization problem with task offloading decision given. Through mathematical manipulations, the non-convex problem is transformed to be a convex one, which is shown to be solvable only with the simple Golden search method. In fast fading channels, optimal online policies depending on instant channel state are derived even though they are entangled. In addition, it is proved that the derived policy will converge to the offline policy when channel coherence time is low, which can help to save extra computation complexity. Numerical results verify the correctness of our analysis and the effectiveness of our proposed strategies over existing methods. △ Less

Submitted 29 December, 2021; v1 submitted 25 November, 2020; originally announced November 2020.

AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?

Authors: Jun Ma, Yao Zhang, Song Gu, Cheng Zhu, Cheng Ge, Yichi Zhang, Xingle An, Congcong Wang, Qiyuan Wang, Xin Liu, Shucheng Cao, Qi Zhang, Shangqing Liu, Yunpeng Wang, Yuhui Li, Jian He, ** Yang

Abstract: With the unprecedented developments in deep learning, automatic segmentation of main abdominal organs seems to be a solved problem as state-of-the-art (SOTA) methods have achieved comparable results with inter-rater variability on many benchmark datasets. However, most of the existing abdominal datasets only contain single-center, single-phase, single-vendor, or single-disease cases, and it is unc… ▽ More With the unprecedented developments in deep learning, automatic segmentation of main abdominal organs seems to be a solved problem as state-of-the-art (SOTA) methods have achieved comparable results with inter-rater variability on many benchmark datasets. However, most of the existing abdominal datasets only contain single-center, single-phase, single-vendor, or single-disease cases, and it is unclear whether the excellent performance can generalize on diverse datasets. This paper presents a large and diverse abdominal CT organ segmentation dataset, termed AbdomenCT-1K, with more than 1000 (1K) CT scans from 12 medical centers, including multi-phase, multi-vendor, and multi-disease cases. Furthermore, we conduct a large-scale study for liver, kidney, spleen, and pancreas segmentation and reveal the unsolved segmentation problems of the SOTA methods, such as the limited generalization ability on distinct medical centers, phases, and unseen diseases. To advance the unsolved problems, we further build four organ segmentation benchmarks for fully supervised, semi-supervised, weakly supervised, and continual learning, which are currently challenging and active research topics. Accordingly, we develop a simple and effective method for each benchmark, which can be used as out-of-the-box methods and strong baselines. We believe the AbdomenCT-1K dataset will promote future in-depth research towards clinical applicable abdominal organ segmentation methods. The datasets, codes, and trained models are publicly available at https://github.com/JunMa11/AbdomenCT-1K. △ Less

Submitted 20 July, 2021; v1 submitted 28 October, 2020; originally announced October 2020.

Comments: 20 pages

Partial FC: Training 10 Million Identities on a Single Machine

Authors: Xiang An, Xuhan Zhu, Yang Xiao, Lan Wu, Ming Zhang, Yuan Gao, Bin Qin, Debing Zhang, Ying Fu

Abstract: Face recognition has been an active and vital topic among computer vision community for a long time. Previous researches mainly focus on loss functions used for facial feature extraction network, among which the improvements of softmax-based loss functions greatly promote the performance of face recognition. However, the contradiction between the drastically increasing number of face identities an… ▽ More Face recognition has been an active and vital topic among computer vision community for a long time. Previous researches mainly focus on loss functions used for facial feature extraction network, among which the improvements of softmax-based loss functions greatly promote the performance of face recognition. However, the contradiction between the drastically increasing number of face identities and the shortage of GPU memories is gradually becoming irreconcilable. In this paper, we thoroughly analyze the optimization goal of softmax-based loss functions and the difficulty of training massive identities. We find that the importance of negative classes in softmax function in face representation learning is not as high as we previously thought. The experiment demonstrates no loss of accuracy when training with only 10\% randomly sampled classes for the softmax-based loss functions, compared with training with full classes using state-of-the-art models on mainstream benchmarks. We also implement a very efficient distributed sampling algorithm, taking into account model accuracy and training efficiency, which uses only eight NVIDIA RTX2080Ti to complete classification tasks with tens of millions of identities. The code of this paper has been made available https://github.com/deepinsight/insightface/tree/master/recognition/partial_fc. △ Less

Submitted 23 January, 2021; v1 submitted 11 October, 2020; originally announced October 2020.

Comments: 8 pages, 9 figures

On Symmetric Rectilinear Matrix Partitioning

Authors: Abdurrahman Yaşar, Muhammed Fatih Balin, Xiao**g An, Kaan Sancak, Ümit V. Çatalyürek

Abstract: Even distribution of irregular workload to processing units is crucial for efficient parallelization in many applications. In this work, we are concerned with a spatial partitioning called rectilinear partitioning (also known as generalized block distribution) of sparse matrices. More specifically, in this work, we address the problem of symmetric rectilinear partitioning of a square matrix. By sy… ▽ More Even distribution of irregular workload to processing units is crucial for efficient parallelization in many applications. In this work, we are concerned with a spatial partitioning called rectilinear partitioning (also known as generalized block distribution) of sparse matrices. More specifically, in this work, we address the problem of symmetric rectilinear partitioning of a square matrix. By symmetric, we mean the rows and columns of the matrix are identically partitioned yielding a tiling where the diagonal tiles (blocks) will be squares. We first show that the optimal solution to this problem is NP-hard, and we propose four heuristics to solve two different variants of this problem. We present a thorough analysis of the computational complexities of those proposed heuristics. To make the proposed techniques more applicable in real life application scenarios, we further reduce their computational complexities by utilizing effective sparsification strategies together with an efficient sparse prefix-sum data structure. We experimentally show the proposed algorithms are efficient and effective on more than six hundred test matrices. With sparsification, our methods take less than 3 seconds in the Twitter graph on a modern 24 core system and output a solution whose load imbalance is no worse than 1%. △ Less

Submitted 16 September, 2020; originally announced September 2020.

Dynamic Functional Connectivity and Graph Convolution Network for Alzheimer's Disease Classification

Authors: Xingwei An, Yutao Zhou, Yang Di, Dong Ming

Abstract: Alzheimer's disease (AD) is the most prevalent form of dementia. Traditional methods cannot achieve efficient and accurate diagnosis of AD. In this paper, we introduce a novel method based on dynamic functional connectivity (dFC) that can effectively capture changes in the brain. We compare and combine four different types of features including amplitude of low-frequency fluctuation (ALFF), region… ▽ More Alzheimer's disease (AD) is the most prevalent form of dementia. Traditional methods cannot achieve efficient and accurate diagnosis of AD. In this paper, we introduce a novel method based on dynamic functional connectivity (dFC) that can effectively capture changes in the brain. We compare and combine four different types of features including amplitude of low-frequency fluctuation (ALFF), regional homogeneity (ReHo), dFC and the adjacency matrix of different brain structures between subjects. We use graph convolution network (GCN) which consider the similarity of brain structure between patients to solve the classification problem of non-Euclidean domains. The proposed method's accuracy and the area under the receiver operating characteristic curve achieved 91.3% and 98.4%. This result demonstrated that our proposed method can be used for detecting AD. △ Less

Submitted 24 June, 2020; originally announced June 2020.

Towards Data-Efficient Learning: A Benchmark for COVID-19 CT Lung and Infection Segmentation

Authors: Jun Ma, Yixin Wang, Xingle An, Cheng Ge, Ziqi Yu, Jianan Chen, Qiongjie Zhu, Guoqiang Dong, Jian He, Zhiqiang He, Yuntao Zhu, Ziwei Nie, ** Yang

Abstract: Purpose: Accurate segmentation of lung and infection in COVID-19 CT scans plays an important role in the quantitative management of patients. Most of the existing studies are based on large and private annotated datasets that are impractical to obtain from a single institution, especially when radiologists are busy fighting the coronavirus disease. Furthermore, it is hard to compare current COVID-… ▽ More Purpose: Accurate segmentation of lung and infection in COVID-19 CT scans plays an important role in the quantitative management of patients. Most of the existing studies are based on large and private annotated datasets that are impractical to obtain from a single institution, especially when radiologists are busy fighting the coronavirus disease. Furthermore, it is hard to compare current COVID-19 CT segmentation methods as they are developed on different datasets, trained in different settings, and evaluated with different metrics. Methods: To promote the development of data-efficient deep learning methods, in this paper, we built three benchmarks for lung and infection segmentation based on 70 annotated COVID-19 cases, which contain current active research areas, e.g., few-shot learning, domain generalization, and knowledge transfer. For a fair comparison among different segmentation methods, we also provide standard training, validation and testing splits, evaluation metrics and, the corresponding code. Results: Based on the state-of-the-art network, we provide more than 40 pre-trained baseline models, which not only serve as out-of-the-box segmentation tools but also save computational time for researchers who are interested in COVID-19 lung and infection segmentation. We achieve average Dice Similarity Coefficient (DSC) scores of 97.3\%, 97.7\%, and 67.3\% and average Normalized Surface Dice (NSD) scores of 90.6\%, 91.4\%, and 70.0\% for left lung, right lung, and infection, respectively. Conclusions: To the best of our knowledge, this work presents the first data-efficient learning benchmark for medical image segmentation and the largest number of pre-trained models up to now. All these resources are publicly available, and our work lays the foundation for promoting the development of deep learning methods for efficient COVID-19 CT segmentation with limited data. △ Less

Submitted 3 December, 2020; v1 submitted 26 April, 2020; originally announced April 2020.

Comments: accepted for publication in Medical Physics

XTDrone: A Customizable Multi-Rotor UAVs Simulation Platform

Authors: Kun Xiao, Shaochang Tan, Guohui Wang, Xueyan An, Xiang Wang, Xiangke Wang

Abstract: A customizable multi-rotor UAVs simulation platform based on ROS, Gazebo and PX4 is presented. The platform, which is called XTDrone, integrates dynamic models, sensor models, control algorithm, state estimation algorithm, and 3D scenes. The platform supports multi UAVs and other robots. The platform is modular and each module can be modified, which means that users can test its own algorithm, suc… ▽ More A customizable multi-rotor UAVs simulation platform based on ROS, Gazebo and PX4 is presented. The platform, which is called XTDrone, integrates dynamic models, sensor models, control algorithm, state estimation algorithm, and 3D scenes. The platform supports multi UAVs and other robots. The platform is modular and each module can be modified, which means that users can test its own algorithm, such as SLAM, object detection, motion planning, attitude control, multi-UAV cooperation, and cooperation with other robots on the platform. The platform runs in lockstep, so the simulation speed can be adjusted according to the computer performance. In this paper, two cases, evaluating different visual SLAM algorithm and realizing UAV formation, are shown to demonstrate how the platform works. △ Less

Submitted 18 August, 2020; v1 submitted 21 March, 2020; originally announced March 2020.

Comments: 8 pages, 15 figures.For the source code see https://github.com/robin-shaun/XTDrone

EmoSense: Computational Intelligence Driven Emotion Sensing via Wireless Channel Data

Authors: Yu Gu, Yantong Wang, Tao Liu, Yusheng Ji, Zhi Liu, Peng Li, Xiaoyan Wang, Xin An, Fuji Ren

Abstract: Emotion is well-recognized as a distinguished symbol of human beings, and it plays a crucial role in our daily lives. Existing vision-based or sensor-based solutions are either obstructive to use or rely on specialized hardware, hindering their applicability. This paper introduces EmoSense, a first-of-its-kind wireless emotion sensing system driven by computational intelligence. The basic methodol… ▽ More Emotion is well-recognized as a distinguished symbol of human beings, and it plays a crucial role in our daily lives. Existing vision-based or sensor-based solutions are either obstructive to use or rely on specialized hardware, hindering their applicability. This paper introduces EmoSense, a first-of-its-kind wireless emotion sensing system driven by computational intelligence. The basic methodology is to explore the physical expression of emotions from wireless channel response via data mining. The design and implementation of EmoSense {face} two major challenges: extracting physical expression from wireless channel data and recovering emotion from the corresponding physical expression. For the former, we present a Fresnel zone based theoretical model depicting the fingerprint of the physical expression on channel response. For the latter, we design an efficient computational intelligence driven mechanism to recognize emotion from the corresponding fingerprints. We prototyped EmoSense on the commodity WiFi infrastructure and compared it with main-stream sensor-based and vision-based approaches in the real-world scenario. The numerical study over $3360$ cases confirms that EmoSense achieves a comparable performance to the vision-based and sensor-based rivals under different scenarios. EmoSense only leverages the low-cost and prevalent WiFi infrastructures and thus constitutes a tempting solution for emotion sensing. △ Less

Submitted 27 August, 2019; originally announced August 2019.

Comments: 11 pages, accepted by IEEE Transactions On Emerging Topics In Computational Intelligence

SwitchAgg:A Further Step Towards In-Network Computation

Authors: Fan Yang, Zhan Wang, Xiaoxiao Ma, Guojun Yuan, Xuejun An

Abstract: Many distributed applications adopt a partition/aggregation pattern to achieve high performance and scalability. The aggregation process, which usually takes a large portion of the overall execution time, incurs large amount of network traffic and bottlenecks the system performance. To reduce network traffic,some researches take advantage of network devices to commit innetwork aggregation. However… ▽ More Many distributed applications adopt a partition/aggregation pattern to achieve high performance and scalability. The aggregation process, which usually takes a large portion of the overall execution time, incurs large amount of network traffic and bottlenecks the system performance. To reduce network traffic,some researches take advantage of network devices to commit innetwork aggregation. However, these approaches use either special topology or middle-boxes, which cannot be easily deployed in current datacenters. The emerging programmable RMT switch brings us new opportunities to implement in-network computation task. However, we argue that the architecture of RMT switch is not suitable for in-network aggregation since it is designed primarily for implementing traditional network functions. In this paper, we first give a detailed analysis of in-network aggregation, and point out the key factor that affects the data reduction ratio. We then propose SwitchAgg, which is an innetwork aggregation system that is compatible with current datacenter infrastructures. We also evaluate the performance improvement we have gained from SwitchAgg. Our results show that, SwitchAgg can process data aggregation tasks at line rate and gives a high data reduction rate, which helps us to cut down network traffic and alleviate pressure on server CPU. In the system performance test, the job-completion-time can be reduced as much as 50% △ Less

Submitted 27 March, 2019; originally announced April 2019.

DarwinML: A Graph-based Evolutionary Algorithm for Automated Machine Learning

Authors: Fei Qi, Zhaohui Xia, Gaoyang Tang, Hang Yang, Yu Song, Guangrui Qian, Xiong An, Chunhuan Lin, Guangming Shi

Abstract: As an emerging field, Automated Machine Learning (AutoML) aims to reduce or eliminate manual operations that require expertise in machine learning. In this paper, a graph-based architecture is employed to represent flexible combinations of ML models, which provides a large searching space compared to tree-based and stacking-based architectures. Based on this, an evolutionary algorithm is proposed… ▽ More As an emerging field, Automated Machine Learning (AutoML) aims to reduce or eliminate manual operations that require expertise in machine learning. In this paper, a graph-based architecture is employed to represent flexible combinations of ML models, which provides a large searching space compared to tree-based and stacking-based architectures. Based on this, an evolutionary algorithm is proposed to search for the best architecture, where the mutation and heredity operators are the key for architecture evolution. With Bayesian hyper-parameter optimization, the proposed approach can automate the workflow of machine learning. On the PMLB dataset, the proposed approach shows the state-of-the-art performance compared with TPOT, Autostacker, and auto-sklearn. Some of the optimized models are with complex structures which are difficult to obtain in manual design. △ Less

Submitted 20 November, 2018; originally announced January 2019.

Comments: 8 pages, 7 figures, 3 tables

Context Awareness in Next Generation of Mobile Core Networks

Authors: Clarissa Cassales Marquezan, Kashif Mahmood, Anastasios Zafeiropoulos, Renan Krishna, Xiaofeng Huang, Xueli An, Daniel Corujo, Filipe Leitão, Maria Lema Rosas, Hans Einsiedler

Abstract: Context awareness is an important enabler for next generation of Mobile Core Networks (MCN). However there exist a number of challenges in this regard. For example how to develop a framework which 1) is able to generate context richer than what is available today; 2) allows reusability of context across the network; 3) provides a mechanism for exposing context to third parties; and 4) can bring to… ▽ More Context awareness is an important enabler for next generation of Mobile Core Networks (MCN). However there exist a number of challenges in this regard. For example how to develop a framework which 1) is able to generate context richer than what is available today; 2) allows reusability of context across the network; 3) provides a mechanism for exposing context to third parties; and 4) can bring together "big data" for mobile core network optimization. In this work, we introduce a context awareness framework addressing the aforementioned challenges but also taking into account the 3GPP standardization activities related to context awareness in MCN. Within this framework we propose Context Generation and Handling Function (CGHF) which generates rich context by processing information from various sources and then handles its distribution through an efficient publish subscribe mechanism. In addition we provide examples where context can be used to optimize control plane decision making. While the focus of this work is on the use of context for MCN, we still believe such context can be also used by applications (at the edge as well as in data centers) and third party services to improve their operations and providing new unforeseen services. △ Less

Submitted 16 November, 2016; originally announced November 2016.

Comments: 15 pages

MSC Class: 68M10

End-to-End Architecture Modularisation and Slicing for Next Generation Networks

Authors: Xueli An, Riccardo Trivisonno, Hans Einsiedler, Dirk von Hugo, Kay Haensge, Xiaofeng Huang, Qing Shen, Daniel Corujo, Kashif Mahmood, Dirk Trossen, Marco Liebsch, Filipe Leitao, Cao-Thanh Phan, Frederic Klamm

Abstract: The journey towards the deployment of next generation networks has recently accelerated, driven by the joint effort of research and standards organisations. Despite this fact, the overall picture is still unclear as prioritization and understanding on several key concepts are not yet agreed by major vendors and network providers. Network Slicing is one of the central topics of the debate, and it i… ▽ More The journey towards the deployment of next generation networks has recently accelerated, driven by the joint effort of research and standards organisations. Despite this fact, the overall picture is still unclear as prioritization and understanding on several key concepts are not yet agreed by major vendors and network providers. Network Slicing is one of the central topics of the debate, and it is expected to become the key feature of next generation networks, providing the flexibility required to support the variety of 5G use cases and business. Network slices are seen as network operator business, offering the possibility to provide flexible services and even infrastructures to vertical industries and classical Telco customers alike. Another key ingredient is the Architecture Modularisation concept, discussed in this paper and regarded by the authors as the essential design principle to build a flexible network architecture natively supporting Network Slicing. According to this concept, conventional monolithic network functions, often corresponding to physical network elements in the existing systems, are to split into basic building blocks defined with the proper granularity, allowing the definition of different logical architectures (i.e. different Network Slices). In this paper, we further discuss a modularisation methodology as a criteria to define the right set of basic building blocks. Defined through this proposed methodology, the set of basic building blocks and the relating interfacing model are discussed. The paper concludes by proposing a modular 5G network architecture as candidate for next generation network standards. △ Less

Submitted 2 November, 2016; originally announced November 2016.

Comments: 13 pages, 5 Figures

MSC Class: 68M10

Generalized Haar Filter based Deep Networks for Real-Time Object Detection in Traffic Scene

Authors: Keyu Lu, Jian Li, Xiang**g An, Hangen He

Abstract: Vision-based object detection is one of the fundamental functions in numerous traffic scene applications such as self-driving vehicle systems and advance driver assistance systems (ADAS). However, it is also a challenging task due to the diversity of traffic scene and the storage, power and computing source limitations of the platforms for traffic scene applications. This paper presents a generali… ▽ More Vision-based object detection is one of the fundamental functions in numerous traffic scene applications such as self-driving vehicle systems and advance driver assistance systems (ADAS). However, it is also a challenging task due to the diversity of traffic scene and the storage, power and computing source limitations of the platforms for traffic scene applications. This paper presents a generalized Haar filter based deep network which is suitable for the object detection tasks in traffic scene. In this approach, we first decompose a object detection task into several easier local regression tasks. Then, we handle the local regression tasks by using several tiny deep networks which simultaneously output the bounding boxes, categories and confidence scores of detected objects. To reduce the consumption of storage and computing resources, the weights of the deep networks are constrained to the form of generalized Haar filter in training phase. Additionally, we introduce the strategy of sparse windows generation to improve the efficiency of the algorithm. Finally, we perform several experiments to validate the performance of our proposed approach. Experimental results demonstrate that the proposed approach is both efficient and effective in traffic scene compared with the state-of-the-art. △ Less

Submitted 30 October, 2016; originally announced October 2016.

Visual Saliency Based on Scale-Space Analysis in the Frequency Domain

Authors: Jian Li, Martin Levine, Xiang**g An, Xin Xu, Hangen He

Abstract: We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of {\it non-saliency}. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale… ▽ More We address the issue of visual saliency from three perspectives. First, we consider saliency detection as a frequency domain analysis problem. Second, we achieve this by employing the concept of {\it non-saliency}. Third, we simultaneously consider the detection of salient regions of different size. The paper proposes a new bottom-up paradigm for detecting visual saliency, characterized by a scale-space analysis of the amplitude spectrum of natural images. We show that the convolution of the {\it image amplitude spectrum} with a low-pass Gaussian kernel of an appropriate scale is equivalent to such an image saliency detector. The saliency map is obtained by reconstructing the 2-D signal using the original phase and the amplitude spectrum, filtered at a scale selected by minimizing saliency map entropy. A Hypercomplex Fourier Transform performs the analysis in the frequency domain. Using available databases, we demonstrate experimentally that the proposed model can predict human fixation data. We also introduce a new image database and use it to show that the saliency detector can highlight both small and large salient regions, as well as inhibit repeated distractors in cluttered images. In addition, we show that it is able to predict salient regions on which people focus their attention. △ Less

Submitted 6 May, 2016; originally announced May 2016.

Journal ref: Pattern Analysis and Machine Intelligence, IEEE Transactions on 35.4 (2013): 996-1010

Optimized Password Recovery for Encrypted RAR on GPUs

Authors: Xiao**g An, Haojun Zhao, Lulu Ding, Zhongrui Fan, Hanyue Wang

Abstract: RAR uses classic symmetric encryption algorithm SHA-1 hashing and AES algorithm for encryption, and the only method of password recovery is brute force, which is very time-consuming. In this paper, we present an approach using GPUs to speed up the password recovery process. However, because the major calculation and time-consuming part, SHA-1 hashing, is hard to be parallelized, so this paper adop… ▽ More RAR uses classic symmetric encryption algorithm SHA-1 hashing and AES algorithm for encryption, and the only method of password recovery is brute force, which is very time-consuming. In this paper, we present an approach using GPUs to speed up the password recovery process. However, because the major calculation and time-consuming part, SHA-1 hashing, is hard to be parallelized, so this paper adopts coarse granularity parallel. That is, one GPU thread is responsible for the validation of one password. We mainly use three optimization methods to optimize this parallel version: asynchronous parallel between CPU and GPU, redundant calculations and conditional statements reduction, and the usage of registers optimization. Experiment result shows that the final version reaches 43~57 times speedup on an AMD FirePro W8000 GPU, compared to a well-optimized serial version on Intel Core i5 CPU. △ Less

Submitted 28 May, 2015; originally announced May 2015.

Towards Power Efficient MAC Protocol for In-Body and On-Body Sensor Networks

Authors: Sana Ullah, Xizhi An, Kyung Sup Kwak

Abstract: This paper presents an empirical discussion on the design and implementation of a power-efficient Medium Access Control (MAC) protocol for in-body and on-body sensor networks. We analyze the performance of a beacon-enabled IEEE 802.15.4, PB-TDMA, and S-MAC protocols for on-body sensor networks. We further present a Traffic Based Wakeup Mechanism that utilizes the traffic patterns of the BAN Node… ▽ More This paper presents an empirical discussion on the design and implementation of a power-efficient Medium Access Control (MAC) protocol for in-body and on-body sensor networks. We analyze the performance of a beacon-enabled IEEE 802.15.4, PB-TDMA, and S-MAC protocols for on-body sensor networks. We further present a Traffic Based Wakeup Mechanism that utilizes the traffic patterns of the BAN Nodes (BNs) to accommodate the entire BSN traffic. To enable a logical connection between different BNs working on different frequency bands, a method called Bridging function is proposed. The Bridging function integrates all BNs working on different bands into a complete BSN. △ Less

Submitted 2 February, 2010; v1 submitted 8 November, 2009; originally announced November 2009.

Comments: 11 pages, 5 figures, 3 tables,KES AMSTA 09, LNAI 5559, pp.335-345, Uppsala, June 2009. arXiv admin note: text overlap with arXiv:0911.1507