Showing 51–100 of 1,018 results for author: Huang, B

Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentation

Authors: Guanlong Jiao, Chenyangguang Zhang, Haonan Yin, Yu Mo, Biqing Huang, Hui Pan, Yi Luo, **gxian Liu

Abstract: Domain generalized semantic segmentation is an essential computer vision task, for which models only leverage source data to learn the capability of generalized semantic segmentation towards the unseen target domains. Previous works typically address this challenge by global style randomization or feature regularization. In this paper, we argue that given the observation that different local seman… ▽ More Domain generalized semantic segmentation is an essential computer vision task, for which models only leverage source data to learn the capability of generalized semantic segmentation towards the unseen target domains. Previous works typically address this challenge by global style randomization or feature regularization. In this paper, we argue that given the observation that different local semantic regions perform different visual characteristics from the source domain to the target domain, methods focusing on global operations are hard to capture such regional discrepancies, thus failing to construct domain-invariant representations with the consistency from local to global level. Therefore, we propose the Semantic-Rearrangement-based Multi-Level Alignment (SRMA) to overcome this problem. SRMA first incorporates a Semantic Rearrangement Module (SRM), which conducts semantic region randomization to enhance the diversity of the source domain sufficiently. A Multi-Level Alignment module (MLA) is subsequently proposed with the help of such diversity to establish the global-regional-local consistent domain-invariant representations. By aligning features across randomized samples with domain-neutral knowledge at multiple levels, SRMA provides a more robust way to handle the source-target domain gap. Extensive experiments demonstrate the superiority of SRMA over the current state-of-the-art works on various benchmarks. △ Less

Submitted 21 April, 2024; originally announced April 2024.

High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces

Authors: Baoru Huang, Yida Wang, Anh Nguyen, Daniel Elson, Francisco Vasconcelos, Danail Stoyanov

Abstract: In surgical oncology, screening colonoscopy plays a pivotal role in providing diagnostic assistance, such as biopsy, and facilitating surgical navigation, particularly in polyp detection. Computer-assisted endoscopic surgery has recently gained attention and amalgamated various 3D computer vision techniques, including camera localization, depth estimation, surface reconstruction, etc. Neural Radia… ▽ More In surgical oncology, screening colonoscopy plays a pivotal role in providing diagnostic assistance, such as biopsy, and facilitating surgical navigation, particularly in polyp detection. Computer-assisted endoscopic surgery has recently gained attention and amalgamated various 3D computer vision techniques, including camera localization, depth estimation, surface reconstruction, etc. Neural Radiance Fields (NeRFs) and Neural Implicit Surfaces (NeuS) have emerged as promising methodologies for deriving accurate 3D surface models from sets of registered images, addressing the limitations of existing colon reconstruction approaches stemming from constrained camera movement. However, the inadequate tissue texture representation and confused scale problem in monocular colonoscopic image reconstruction still impede the progress of the final rendering results. In this paper, we introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map. Notably, we pioneered the exploration of utilizing only one frame depth map in photorealistic reconstruction and neural rendering applications while this single depth map can be easily obtainable from other monocular depth estimation networks with an object scale. Through rigorous experimentation and validation on phantom imagery, our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface. This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions. △ Less

Submitted 20 April, 2024; originally announced April 2024.

Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption

Authors: Buzhen Huang, Chen Li, Chongyang Xu, Liang Pan, Yangang Wang, Gim Hee Lee

Abstract: Existing multi-person human reconstruction approaches mainly focus on recovering accurate poses or avoiding penetration, but overlook the modeling of close interactions. In this work, we tackle the task of reconstructing closely interactive humans from a monocular video. The main challenge of this task comes from insufficient visual information caused by depth ambiguity and severe inter-person occ… ▽ More Existing multi-person human reconstruction approaches mainly focus on recovering accurate poses or avoiding penetration, but overlook the modeling of close interactions. In this work, we tackle the task of reconstructing closely interactive humans from a monocular video. The main challenge of this task comes from insufficient visual information caused by depth ambiguity and severe inter-person occlusion. In view of this, we propose to leverage knowledge from proxemic behavior and physics to compensate the lack of visual information. This is based on the observation that human interaction has specific patterns following the social proxemics. Specifically, we first design a latent representation based on Vector Quantised-Variational AutoEncoder (VQ-VAE) to model human interaction. A proxemics and physics guided diffusion model is then introduced to denoise the initial distribution. We design the diffusion model as dual branch with each branch representing one individual such that the interaction can be modeled via cross attention. With the learned priors of VQ-VAE and physical constraint as the additional information, our proposed approach is capable of estimating accurate poses that are also proxemics and physics plausible. Experimental results on Hi4D, 3DPW, and CHI3D demonstrate that our method outperforms existing approaches. The code is available at \url{https://github.com/boycehbz/HumanInteraction}. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Comments: CVPR2024

A Progressive Framework of Vision-language Knowledge Distillation and Alignment for Multilingual Scene

Authors: Wenbo Zhang, Yifan Zhang, Jianfeng Lin, Binqiang Huang, **lu Zhang, Wenhao Yu

Abstract: Pre-trained vision-language (V-L) models such as CLIP have shown excellent performance in many downstream cross-modal tasks. However, most of them are only applicable to the English context. Subsequent research has focused on this problem and proposed improved models, such as CN-CLIP and AltCLIP, to facilitate their applicability to Chinese and even other languages. Nevertheless, these models suff… ▽ More Pre-trained vision-language (V-L) models such as CLIP have shown excellent performance in many downstream cross-modal tasks. However, most of them are only applicable to the English context. Subsequent research has focused on this problem and proposed improved models, such as CN-CLIP and AltCLIP, to facilitate their applicability to Chinese and even other languages. Nevertheless, these models suffer from high latency and a large memory footprint in inference, which limits their further deployment on resource-constrained edge devices. In this work, we propose a conceptually simple yet effective multilingual CLIP Compression framework and train a lightweight multilingual vision-language model, called DC-CLIP, for both Chinese and English context. In this framework, we collect high-quality Chinese and English text-image pairs and design two training stages, including multilingual vision-language feature distillation and alignment. During the first stage, lightweight image/text student models are designed to learn robust visual/multilingual textual feature representation ability from corresponding teacher models, respectively. Subsequently, the multilingual vision-language alignment stage enables effective alignment of visual and multilingual textual features to further improve the model's multilingual performance. Comprehensive experiments in zero-shot image classification, conducted based on the ELEVATER benchmark, showcase that DC-CLIP achieves superior performance in the English context and competitive performance in the Chinese context, even with less training data, when compared to existing models of similar parameter magnitude. The evaluation demonstrates the effectiveness of our designed training mechanism. △ Less

Submitted 17 April, 2024; originally announced April 2024.

A universal strategy for decoupling stiffness and extensibility of polymer networks

Authors: Baiqiang Huang, Shifeng Nian, Li-Heng Cai

Abstract: Since the invention of polymer networks in the 19th century (e.g., crosslinked natural rubber by Goodyear), it has been a dogma that stiffer networks are less stretchable, a trade-off inherent to the molecular nature of polymer network strands. Here, we report a universal strategy for decoupling the stiffness and extensibility of single-network elastomers. Instead of using linear polymers as netwo… ▽ More Since the invention of polymer networks in the 19th century (e.g., crosslinked natural rubber by Goodyear), it has been a dogma that stiffer networks are less stretchable, a trade-off inherent to the molecular nature of polymer network strands. Here, we report a universal strategy for decoupling the stiffness and extensibility of single-network elastomers. Instead of using linear polymers as network strands, we use foldable bottlebrush polymers, which feature a collapsed backbone grafted with many linear side chains. Upon elongation, the collapsed backbone unfolds to release stored length, enabling remarkable extensibility. By contrast, the network elastic modulus is inversely proportional to the network strand mass and is determined by the side chains. We validate this concept by creating a series of unentangled single-network elastomers with nearly constant Young's modulus (30 kPa) while increasing tensile breaking strain by 40-fold, from 20% to 800%. We show that this strategy applies to networks of different polymer species and topologies. Our discovery opens an avenue for develo** polymer networks with extraordinary mechanical properties. △ Less

Submitted 10 May, 2024; v1 submitted 14 April, 2024; originally announced April 2024.

Comments: 4 figures; 10 extended figures; 1 extended table

Ferroelectrovalley in Two-Dimensional Multiferroic Lattices

Authors: Jiangyu Zhao, Yangyang Feng, Ying Dai, Baibiao Huang, Yandong Ma

Abstract: Engineering valley index is essential and highly sought for valley physics, but currently it is exclusively based on the paradigm of the challenging ferrovalley with spin-orientation reversal under magnetic field. Here, an alternative strategy, i.e., the so-called ferroelectrovalley, is proposed to tackle the insurmountable spin-orientation reversal, which reveres valley index with the feasible fe… ▽ More Engineering valley index is essential and highly sought for valley physics, but currently it is exclusively based on the paradigm of the challenging ferrovalley with spin-orientation reversal under magnetic field. Here, an alternative strategy, i.e., the so-called ferroelectrovalley, is proposed to tackle the insurmountable spin-orientation reversal, which reveres valley index with the feasible ferroelectricity. Using symmetry arguments and tight-binding model, the C_2 rotation is unveiled to be able to take the place of time reversal for operating valley index in two-dimensional multiferroic kagome lattices, which enables the ferroelectricity-engineered valley index, thereby generating the concept of ferroelectrovalley. Based on first-principles calculations, this concept is further demonstrated in the breathing kagome lattice of single-layer Ti3Br8, wherein ferroelectricity couples the breathing process. These findings open a new direction for valleytronics and two-dimensional materials research. △ Less

Submitted 13 April, 2024; originally announced April 2024.

Xiwu: A Basis Flexible and Learnable LLM for High Energy Physics

Authors: Zhengde Zhang, Yiyu Zhang, Haodong Yao, Jianwen Luo, Rui Zhao, Bo Huang, Jiameng Zhao, Yipu Liao, Ke Li, Lina Zhao, Jun Cao, Fazhi Qi, Changzheng Yuan

Abstract: Large Language Models (LLMs) are undergoing a period of rapid updates and changes, with state-of-the-art (SOTA) model frequently being replaced. When applying LLMs to a specific scientific field, it's challenging to acquire unique domain knowledge while kee** the model itself advanced. To address this challenge, a sophisticated large language model system named as Xiwu has been developed, allowi… ▽ More Large Language Models (LLMs) are undergoing a period of rapid updates and changes, with state-of-the-art (SOTA) model frequently being replaced. When applying LLMs to a specific scientific field, it's challenging to acquire unique domain knowledge while kee** the model itself advanced. To address this challenge, a sophisticated large language model system named as Xiwu has been developed, allowing you switch between the most advanced foundation models and quickly teach the model domain knowledge. In this work, we will report on the best practices for applying LLMs in the field of high-energy physics (HEP), including: a seed fission technology is proposed and some data collection and cleaning tools are developed to quickly obtain domain AI-Ready dataset; a just-in-time learning system is implemented based on the vector store technology; an on-the-fly fine-tuning system has been developed to facilitate rapid training under a specified foundation model. The results show that Xiwu can smoothly switch between foundation models such as LLaMA, Vicuna, ChatGLM and Grok-1. The trained Xiwu model is significantly outperformed the benchmark model on the HEP knowledge question-and-answering and code generation. This strategy significantly enhances the potential for growth of our model's performance, with the hope of surpassing GPT-4 as it evolves with the development of open-source models. This work provides a customized LLM for the field of HEP, while also offering references for applying LLM to other fields, the corresponding codes are available on Github. △ Less

Submitted 8 April, 2024; originally announced April 2024.

Comments: 15 pages, 8 figures

ACM Class: I.2.7

Topology-engineered orbital Hall effect in two-dimensional ferromagnets

Authors: Zhiqi Chen, Runhan Li, Yingxi Bai, Ning Mao, Mahmoud Zeer, Dongwook Go, Ying Dai, Baibiao Huang, Yuriy Mokrousov, Chengwang Niu

Abstract: Recent advances in manipulation of orbital angular momentum (OAM) within the paradigm of orbitronics present a promising avenue for the design of future electronic devices. In this context, the recently observed orbital Hall effect (OHE) occupies a special place. Here, focusing on both the second-order topological and quantum anomalous Hall insulators in two-dimensional ferromagnets, we demonstrat… ▽ More Recent advances in manipulation of orbital angular momentum (OAM) within the paradigm of orbitronics present a promising avenue for the design of future electronic devices. In this context, the recently observed orbital Hall effect (OHE) occupies a special place. Here, focusing on both the second-order topological and quantum anomalous Hall insulators in two-dimensional ferromagnets, we demonstrate that topological phase transitions present an efficient and straightforward way to engineer the OHE, where the OAM distribution can be controlled by the nature of the band inversion. Using first-principles calculations, we identify Janus RuBrCl and three septuple layers of MnBi$_2$Te$_4$ as experimentally feasible examples of the proposed mechanism of OHE engineering by topology. With our work we open up new possibilities for innovative applications in topological spintronics and orbitronics. △ Less

Submitted 11 April, 2024; originally announced April 2024.

Comments: 4 figures

Toward Holistic Planning and Control Optimization for Dual-Arm Rearrangement

Authors: Kai Gao, Zihe Ye, Duo Zhang, Baichuan Huang, **g** Yu

Abstract: Long-horizon task and motion planning (TAMP) is notoriously difficult to solve, let alone optimally, due to the tight coupling between the interleaved (discrete) task and (continuous) motion planning phases, where each phase on its own is frequently an NP-hard or even PSPACE-hard computational challenge. In this study, we tackle the even more challenging goal of jointly optimizing task and motion… ▽ More Long-horizon task and motion planning (TAMP) is notoriously difficult to solve, let alone optimally, due to the tight coupling between the interleaved (discrete) task and (continuous) motion planning phases, where each phase on its own is frequently an NP-hard or even PSPACE-hard computational challenge. In this study, we tackle the even more challenging goal of jointly optimizing task and motion plans for a real dual-arm system in which the two arms operate in close vicinity to solve highly constrained tabletop multi-object rearrangement problems. Toward that, we construct a tightly integrated planning and control optimization pipeline, Makespan-Optimized Dual-Arm Planner (MODAP) that combines novel sampling techniques for task planning with state-of-the-art trajectory optimization techniques. Compared to previous state-of-the-art, MODAP produces task and motion plans that better coordinate a dual-arm system, delivering significantly improved execution time improvements while simultaneously ensuring that the resulting time-parameterized trajectory conforms to specified acceleration and jerk limits. △ Less

Submitted 10 April, 2024; originally announced April 2024.

Comments: First three authors made equal contributions to this study

CausalBench: A Comprehensive Benchmark for Causal Learning Capability of Large Language Models

Authors: Yu Zhou, Xingyu Wu, Beicheng Huang, Jibin Wu, Liang Feng, Kay Chen Tan

Abstract: Causality reveals fundamental principles behind data distributions in real-world scenarios, and the capability of large language models (LLMs) to understand causality directly impacts their efficacy across explaining outputs, adapting to new evidence, and generating counterfactuals. With the proliferation of LLMs, the evaluation of this capacity is increasingly garnering attention. However, the ab… ▽ More Causality reveals fundamental principles behind data distributions in real-world scenarios, and the capability of large language models (LLMs) to understand causality directly impacts their efficacy across explaining outputs, adapting to new evidence, and generating counterfactuals. With the proliferation of LLMs, the evaluation of this capacity is increasingly garnering attention. However, the absence of a comprehensive benchmark has rendered existing evaluation studies being straightforward, undiversified, and homogeneous. To address these challenges, this paper proposes a comprehensive benchmark, namely CausalBench, to evaluate the causality understanding capabilities of LLMs. Originating from the causal research community, CausalBench encompasses three causal learning-related tasks, which facilitate a convenient comparison of LLMs' performance with classic causal learning algorithms. Meanwhile, causal networks of varying scales and densities are integrated in CausalBench, to explore the upper limits of LLMs' capabilities across task scenarios of varying difficulty. Notably, background knowledge and structured data are also incorporated into CausalBench to thoroughly unlock the underlying potential of LLMs for long-text comprehension and prior information utilization. Based on CausalBench, this paper evaluates nineteen leading LLMs and unveils insightful conclusions in diverse aspects. Firstly, we present the strengths and weaknesses of LLMs and quantitatively explore the upper limits of their capabilities across various scenarios. Meanwhile, we further discern the adaptability and abilities of LLMs to specific structural networks and complex chain of thought structures. Moreover, this paper quantitatively presents the differences across diverse information sources and uncovers the gap between LLMs' capabilities in causal understanding within textual contexts and numerical domains. △ Less

Submitted 9 April, 2024; originally announced April 2024.

Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models

Authors: Beichen Huang, Xingyu Wu, Yu Zhou, Jibin Wu, Liang Feng, Ran Cheng, Kay Chen Tan

Abstract: Large language models (LLMs) have gained widespread popularity and demonstrated exceptional performance not only in natural language processing (NLP) tasks but also in non-linguistic domains. Their potential as artificial general intelligence extends beyond NLP, showcasing promising capabilities in diverse optimization scenarios. Despite this rising trend, whether the integration of LLMs into thes… ▽ More Large language models (LLMs) have gained widespread popularity and demonstrated exceptional performance not only in natural language processing (NLP) tasks but also in non-linguistic domains. Their potential as artificial general intelligence extends beyond NLP, showcasing promising capabilities in diverse optimization scenarios. Despite this rising trend, whether the integration of LLMs into these black-box optimization problems is genuinely beneficial remains unexplored. This paper endeavors to tackle this issue by offering deeper insights into the potential of LLMs in optimization tasks through a comprehensive investigation. Our approach involves a comprehensive evaluation, covering both discrete and continuous optimization problems, aiming to assess the efficacy and distinctive characteristics that LLMs bring to the realm of optimization. Our findings reveal both the limitations and advantages of LLMs in optimization. On one hand, despite consuming the significant power required to run the model, LLMs exhibit subpar performance and lack desirable properties in pure numerical tasks, primarily due to a mismatch between the problem domain and their processing capabilities. On the other hand, although LLMs may not be ideal for traditional numerical optimization, their potential in broader optimization contexts remains promising. LLMs exhibit the ability to solve problems in non-numerical domains and can leverage heuristics from the prompt to enhance their performance. To the best of our knowledge, this work presents the first systematic evaluation of LLMs for numerical optimization, offering a progressive, wide-coverage, and behavioral analysis. Our findings pave the way for a deeper understanding of LLMs' role in optimization and guide future application in diverse scenarios for LLMs. △ Less

Submitted 9 April, 2024; originally announced April 2024.

Lightweight Inference for Forward-Forward Algorithm

Authors: Amin Aminifar, Baichuan Huang, Azra Abtahi, Amir Aminifar

Abstract: The human brain performs tasks with an outstanding energy-efficiency, i.e., with approximately 20 Watts. The state-of-the-art Artificial/Deep Neural Networks (ANN/DNN), on the other hand, have recently been shown to consume massive amounts of energy. The training of these ANNs/DNNs is done almost exclusively based on the back-propagation algorithm, which is known to be biologically implausible. Th… ▽ More The human brain performs tasks with an outstanding energy-efficiency, i.e., with approximately 20 Watts. The state-of-the-art Artificial/Deep Neural Networks (ANN/DNN), on the other hand, have recently been shown to consume massive amounts of energy. The training of these ANNs/DNNs is done almost exclusively based on the back-propagation algorithm, which is known to be biologically implausible. This has led to a new generation of forward-only techniques, including the Forward-Forward algorithm. In this paper, we propose a lightweight inference scheme specifically designed for DNNs trained using the Forward-Forward algorithm. We have evaluated our proposed lightweight inference scheme in the case of the MNIST and CIFAR datasets, as well as two real-world applications, namely, epileptic seizure detection and cardiac arrhythmia classification using wearable technologies, where complexity overheads/energy consumption is a major constraint, and demonstrate its relevance. △ Less

Submitted 14 May, 2024; v1 submitted 8 April, 2024; originally announced April 2024.

On-demand higher-harmonic generation through nonlinear Hall effects in curved nanomembranes

Authors: Botsz Huang, You-Ting Huang, Jan-Chi Yang, Tse-Ming Chen, Ali G. Moghaddam, Ching-Hao Chang

Abstract: The high-order Hall effects, which go beyond the ordinary, unlock more possibilities of electronic transport properties and functionalities. Pioneer works focus on the manufacture of complex nanostructures with low lattice symmetry to produce them. In this paper, we theoretically show that such high-order Hall effects can alternatively be generated by curving a conducting nanomembrane which is hig… ▽ More The high-order Hall effects, which go beyond the ordinary, unlock more possibilities of electronic transport properties and functionalities. Pioneer works focus on the manufacture of complex nanostructures with low lattice symmetry to produce them. In this paper, we theoretically show that such high-order Hall effects can alternatively be generated by curving a conducting nanomembrane which is highly tunable and also enables anisotropy. Its Hall response can be tuned from first to fourth order by simply varying the direction and magnitude of the applied magnetic field. The dominant Hall current frequency can also be altered from zero to double, or even four times that of the applied alternating electric field. This phenomenon is critically dependent on the occurrence of high-order snake orbits associated with the effective magnetic-field dipoles and quadruples induced by the curved geometry. Our results offer pathways for spatially engineering magnetotransport, current rectification, and frequency multiplication in the bent conducting nanomembrane. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: 7+3 pages,5+2 figures

Search for a sub-eV sterile neutrino using Daya Bay's full dataset

Authors: F. P. An, W. D. Bai, A. B. Balantekin, M. Bishai, S. Blyth, G. F. Cao, J. Cao, J. F. Chang, Y. Chang, H. S. Chen, H. Y. Chen, S. M. Chen, Y. Chen, Y. X. Chen, Z. Y. Chen, J. Cheng, Y. C. Cheng, Z. K. Cheng, J. J. Cherwinka, M. C. Chu, J. P. Cummings, O. Dalager, F. S. Deng, X. Y. Ding, Y. Y. Ding , et al. (176 additional authors not shown)

Abstract: This Letter presents results of a search for the mixing of a sub-eV sterile neutrino with three active neutrinos based on the full data sample of the Daya Bay Reactor Neutrino Experiment, collected during 3158 days of detector operation, which contains $5.55 \times 10^{6}$ reactor \anue candidates identified as inverse beta-decay interactions followed by neutron-capture on gadolinium. The analysis… ▽ More This Letter presents results of a search for the mixing of a sub-eV sterile neutrino with three active neutrinos based on the full data sample of the Daya Bay Reactor Neutrino Experiment, collected during 3158 days of detector operation, which contains $5.55 \times 10^{6}$ reactor \anue candidates identified as inverse beta-decay interactions followed by neutron-capture on gadolinium. The analysis benefits from a doubling of the statistics of our previous result and from improvements of several important systematic uncertainties. No significant oscillation due to mixing of a sub-eV sterile neutrino with active neutrinos was found. Exclusion limits are set by both Feldman-Cousins and CLs methods. Light sterile neutrino mixing with $\sin^2 2θ_{14} \gtrsim 0.01$ can be excluded at 95\% confidence level in the region of $0.01$ eV$^2 \lesssim |Δm^{2}_{41}| \lesssim 0.1 $ eV$^2$. This result represents the world-leading constraints in the region of $2 \times 10^{-4}$ eV$^2 \lesssim |Δm^{2}_{41}| \lesssim 0.2 $ eV$^2$. △ Less

Submitted 15 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: 7 pages, 4 figures, 1 table

An OpenStreetMaps based tool to study the energy demand and emissions impact of electrification of medium and heavy-duty freight trucks

Authors: Nawaf Nazir, Bowen Huang, Shant Mahserejian

Abstract: In this paper, we present the mathematical formulation of an OpenStreetMaps (OSM) based tool that compares the costs and emissions of long-haul medium and heavy-duty (M&HD) electric and diesel freight trucks, and determines the spatial distribution of added energy demand due to M&HD EVs. The optimization utilizes a combination of information on routes from OSM, utility rate design data across the… ▽ More In this paper, we present the mathematical formulation of an OpenStreetMaps (OSM) based tool that compares the costs and emissions of long-haul medium and heavy-duty (M&HD) electric and diesel freight trucks, and determines the spatial distribution of added energy demand due to M&HD EVs. The optimization utilizes a combination of information on routes from OSM, utility rate design data across the United States, and freight volume data, to determine these values. In order to deal with the computational complexity of this problem, we formulate the problem as a convex optimization problem that is scalable to a large geographic area. In our analysis, we further evaluate various scenarios of utility rate design (energy charges) and EV penetration rate across different geographic regions and their impact on the operating cost and emissions of the freight trucks. Our approach determines the net emissions reduction benefits of freight electrification by considering the primary energy source in different regions. Such analysis will provide insights to policy makers in designing utility rates for electric vehicle supply equipment (EVSE) operators depending upon the specific geographic region and to electric utilities in deciding infrastructure upgrades based on the spatial distribution of the added energy demand of M&HD EVs. To showcase the results, a case study for the U.S. state of Texas is conducted. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Automated Bi-Fold Weighted Ensemble Algorithms and its Application to Brain Tumor Detection and Classification

Authors: PoTsang B. Huang, Muhammad Rizwan, Mehboob Ali

Abstract: The uncontrolled and unstructured growth of brain cells is known as brain tumor, which has one of the highest mortality rates among diseases from all types of cancers. Due to limited diagnostic and treatment capabilities, they pose significant challenges, especially in third-world countries. Early diagnosis plays a vital role in effectively managing brain tumors and reducing mortality rates. Howev… ▽ More The uncontrolled and unstructured growth of brain cells is known as brain tumor, which has one of the highest mortality rates among diseases from all types of cancers. Due to limited diagnostic and treatment capabilities, they pose significant challenges, especially in third-world countries. Early diagnosis plays a vital role in effectively managing brain tumors and reducing mortality rates. However, the availability of diagnostic methods is hindered by various limitations, including high costs and lengthy result acquisition times, impeding early detection of the disease. In this study, we present two cutting-edge bi-fold weighted voting ensemble models that aim to boost the effectiveness of weighted ensemble methods. These two proposed methods combine the classification outcomes from multiple classifiers and determine the optimal result by selecting the one with the highest probability in the first approach, and the highest weighted prediction in the second technique. These approaches significantly improve the overall performance of weighted ensemble techniques. In the first proposed method, we improve the soft voting technique (SVT) by introducing a novel unsupervised weight calculating schema (UWCS) to enhance its weight assigning capability, known as the extended soft voting technique (ESVT). Secondly, we propose a novel weighted method (NWM) by using the proposed UWCS. Both of our approaches incorporate three distinct models: a custom-built CNN, VGG-16, and InceptionResNetV2 which has been trained on publicly available datasets. The effectiveness of our proposed systems is evaluated through blind testing, where exceptional results are achieved. We then establish a comparative analysis of the performance of our proposed methods with that of SVT to show their superiority and effectiveness. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Facilitating Reinforcement Learning for Process Control Using Transfer Learning: Perspectives

Authors: Runze Lin, Junghui Chen, Lei Xie, Hongye Su, Biao Huang

Abstract: This paper provides insights into deep reinforcement learning (DRL) for process control from the perspective of transfer learning. We analyze the challenges of applying DRL in the field of process industries and the necessity of introducing transfer learning. Furthermore, recommendations and prospects are provided for future research directions on how transfer learning can be integrated with DRL t… ▽ More This paper provides insights into deep reinforcement learning (DRL) for process control from the perspective of transfer learning. We analyze the challenges of applying DRL in the field of process industries and the necessity of introducing transfer learning. Furthermore, recommendations and prospects are provided for future research directions on how transfer learning can be integrated with DRL to empower process control. △ Less

Submitted 1 May, 2024; v1 submitted 30 March, 2024; originally announced April 2024.

Comments: Final Version of Asian Control Conference (ASCC 2024)

SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Control

Authors: Binyuan Huang, Yuqing Wen, Yucheng Zhao, Yaosi Hu, Yingfei Liu, Fan Jia, Weixin Mao, Tiancai Wang, Chi Zhang, Chang Wen Chen, Zhenzhong Chen, Xiangyu Zhang

Abstract: Autonomous driving progress relies on large-scale annotated datasets. In this work, we explore the potential of generative models to produce vast quantities of freely-labeled data for autonomous driving applications and present SubjectDrive, the first model proven to scale generative data production in a way that could continuously improve autonomous driving applications. We investigate the impact… ▽ More Autonomous driving progress relies on large-scale annotated datasets. In this work, we explore the potential of generative models to produce vast quantities of freely-labeled data for autonomous driving applications and present SubjectDrive, the first model proven to scale generative data production in a way that could continuously improve autonomous driving applications. We investigate the impact of scaling up the quantity of generative data on the performance of downstream perception models and find that enhancing data diversity plays a crucial role in effectively scaling generative data production. Therefore, we have developed a novel model equipped with a subject control mechanism, which allows the generative model to leverage diverse external data sources for producing varied and useful data. Extensive evaluations confirm SubjectDrive's efficacy in generating scalable autonomous driving training data, marking a significant step toward revolutionizing data production methods in this field. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Comments: Project page: https://subjectdrive.github.io/

Taming Lookup Tables for Efficient Image Retouching

Authors: Sidi Yang, Binxiao Huang, Mingdeng Cao, Yatai Ji, Hanzhong Guo, Ngai Wong, Yujiu Yang

Abstract: The widespread use of high-definition screens in edge devices, such as end-user cameras, smartphones, and televisions, is spurring a significant demand for image enhancement. Existing enhancement models often optimize for high performance while falling short of reducing hardware inference time and power consumption, especially on edge devices with constrained computing and storage resources. To th… ▽ More The widespread use of high-definition screens in edge devices, such as end-user cameras, smartphones, and televisions, is spurring a significant demand for image enhancement. Existing enhancement models often optimize for high performance while falling short of reducing hardware inference time and power consumption, especially on edge devices with constrained computing and storage resources. To this end, we propose Image Color Enhancement Lookup Table (ICELUT) that adopts LUTs for extremely efficient edge inference, without any convolutional neural network (CNN). During training, we leverage pointwise (1x1) convolution to extract color information, alongside a split fully connected layer to incorporate global information. Both components are then seamlessly converted into LUTs for hardware-agnostic deployment. ICELUT achieves near-state-of-the-art performance and remarkably low power consumption. We observe that the pointwise network structure exhibits robust scalability, upkee** the performance even with a heavily downsampled 32x32 input image. These enable ICELUT, the first-ever purely LUT-based image enhancer, to reach an unprecedented speed of 0.4ms on GPU and 7ms on CPU, at least one order faster than any CNN solution. Codes are available at https://github.com/Stephen0808/ICELUT. △ Less

Submitted 28 March, 2024; originally announced March 2024.

AAPMT: AGI Assessment Through Prompt and Metric Transformer

Authors: Benhao Huang

Abstract: The emergence of text-to-image models marks a significant milestone in the evolution of AI-generated images (AGIs), expanding their use in diverse domains like design, entertainment, and more. Despite these breakthroughs, the quality of AGIs often remains suboptimal, highlighting the need for effective evaluation methods. These methods are crucial for assessing the quality of images relative to th… ▽ More The emergence of text-to-image models marks a significant milestone in the evolution of AI-generated images (AGIs), expanding their use in diverse domains like design, entertainment, and more. Despite these breakthroughs, the quality of AGIs often remains suboptimal, highlighting the need for effective evaluation methods. These methods are crucial for assessing the quality of images relative to their textual descriptions, and they must accurately mirror human perception. Substantial progress has been achieved in this domain, with innovative techniques such as BLIP and DBCNN contributing significantly. However, recent studies, including AGIQA-3K, reveal a notable discrepancy between current methods and state-of-the-art (SOTA) standards. This gap emphasizes the necessity for a more sophisticated and precise evaluation metric. In response, our objective is to develop a model that could give ratings for metrics, which focuses on parameters like perceptual quality, authenticity, and the correspondence between text and image, that more closely aligns with human perception. In our paper, we introduce a range of effective methods, including prompt designs and the Metric Transformer. The Metric Transformer is a novel structure inspired by the complex interrelationships among various AGI quality metrics. The code is available at https://github.com/huskydoge/CS3324-Digital-Image-Processing/tree/main/Assignment1 △ Less

Submitted 27 March, 2024; originally announced March 2024.

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Authors: Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, Shenghua Gao

Abstract: 3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. However, 3DGS fails to accurately represent surfaces due to the multi-view inconsistent nature of 3D Gaussians. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance… ▽ More 3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. However, 3DGS fails to accurately represent surfaces due to the multi-view inconsistent nature of 3D Gaussians. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. Our key idea is to collapse the 3D volume into a set of 2D oriented planar Gaussian disks. Unlike 3D Gaussians, 2D Gaussians provide view-consistent geometry while modeling surfaces intrinsically. To accurately recover thin surfaces and achieve stable optimization, we introduce a perspective-correct 2D splatting process utilizing ray-splat intersection and rasterization. Additionally, we incorporate depth distortion and normal consistency terms to further enhance the quality of the reconstructions. We demonstrate that our differentiable renderer allows for noise-free and detailed geometry reconstruction while maintaining competitive appearance quality, fast training speed, and real-time rendering. △ Less

Submitted 9 June, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Comments: 13 pages, 12 figures

Identifiable Latent Neural Causal Models

Authors: Yuhang Liu, Zhen Zhang, Dong Gong, Mingming Gong, Biwei Huang, Anton van den Hengel, Kun Zhang, Javen Qinfeng Shi

Abstract: Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. It is particularly good at predictions under unseen distribution shifts, because these shifts can generally be interpreted as consequences of interventions. Hence leveraging {seen} distribution shifts becomes a natural strategy to help identifying causal representations, which in… ▽ More Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. It is particularly good at predictions under unseen distribution shifts, because these shifts can generally be interpreted as consequences of interventions. Hence leveraging {seen} distribution shifts becomes a natural strategy to help identifying causal representations, which in turn benefits predictions where distributions are previously {unseen}. Determining the types (or conditions) of such distribution shifts that do contribute to the identifiability of causal representations is critical. This work establishes a {sufficient} and {necessary} condition characterizing the types of distribution shifts for identifiability in the context of latent additive noise models. Furthermore, we present partial identifiability results when only a portion of distribution shifts meets the condition. In addition, we extend our findings to latent post-nonlinear causal models. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations. Our algorithm, guided by our underlying theory, has demonstrated outstanding performance across a diverse range of synthetic and real-world datasets. The empirical observations align closely with the theoretical findings, affirming the robustness and effectiveness of our approach. △ Less

Submitted 23 March, 2024; originally announced March 2024.

Temporal-Spatial Object Relations Modeling for Vision-and-Language Navigation

Authors: Bowen Huang, Yanwei Zheng, Chuanlin Lan, Xinpeng Zhao, Yifei Zou, Dongxiao yu

Abstract: Vision-and-Language Navigation (VLN) is a challenging task where an agent is required to navigate to a natural language described location via vision observations. The navigation abilities of the agent can be enhanced by the relations between objects, which are usually learned using internal objects or external datasets. The relationships between internal objects are modeled employing graph convol… ▽ More Vision-and-Language Navigation (VLN) is a challenging task where an agent is required to navigate to a natural language described location via vision observations. The navigation abilities of the agent can be enhanced by the relations between objects, which are usually learned using internal objects or external datasets. The relationships between internal objects are modeled employing graph convolutional network (GCN) in traditional studies. However, GCN tends to be shallow, limiting its modeling ability. To address this issue, we utilize a cross attention mechanism to learn the connections between objects over a trajectory, which takes temporal continuity into account, termed as Temporal Object Relations (TOR). The external datasets have a gap with the navigation environment, leading to inaccurate modeling of relations. To avoid this problem, we construct object connections based on observations from all viewpoints in the navigational environment, which ensures complete spatial coverage and eliminates the gap, called Spatial Object Relations (SOR). Additionally, we observe that agents may repeatedly visit the same location during navigation, significantly hindering their performance. For resolving this matter, we introduce the Turning Back Penalty (TBP) loss function, which penalizes the agent's repetitive visiting behavior, substantially reducing the navigational distance. Experimental results on the REVERIE, SOON, and R2R datasets demonstrate the effectiveness of the proposed method. △ Less

Submitted 16 May, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

Adaptive Target Detection for FDA-MIMO Radar with Training Data in Gaussian noise

Authors: ** Li, Bang Huang, Wen-Qin Wang

Abstract: This paper addresses the problem of detecting a moving target embedded in Gaussian noise with an unknown covariance matrix for frequency diverse array multiple-input multiple-output (FDA-MIMO) radar. To end it, assume that obtaining a set of training data is available. Moreover, we propose three adaptive detectors in accordance with the one-step generalized likelihood ratio test (GLRT), two-step G… ▽ More This paper addresses the problem of detecting a moving target embedded in Gaussian noise with an unknown covariance matrix for frequency diverse array multiple-input multiple-output (FDA-MIMO) radar. To end it, assume that obtaining a set of training data is available. Moreover, we propose three adaptive detectors in accordance with the one-step generalized likelihood ratio test (GLRT), two-step GLRT, and Rao criteria, namely OGLRT, TGLRT, and Rao. The LH adaptive matched filter (LHAMF) detector is also introduced when decomposing the Rao test. Next, all provided detectors have constant false alarm rate (CFAR) properties against the covariance matrix. Besides, the closed-form expressions for false alarm probability (PFA) and detection probability (PD) are derived. Finally, this paper substantiates the correctness of the aforementioned algorithms through numerical simulations. △ Less

Submitted 21 March, 2024; originally announced March 2024.

Data Acquisition via Experimental Design for Decentralized Data Markets

Authors: Charles Lu, Baihe Huang, Sai Praneeth Karimireddy, Praneeth Vepakomma, Michael Jordan, Ramesh Raskar

Abstract: Acquiring high-quality training data is essential for current machine learning models. Data markets provide a way to increase the supply of data, particularly in data-scarce domains such as healthcare, by incentivizing potential data sellers to join the market. A major challenge for a data buyer in such a market is selecting the most valuable data points from a data seller. Unlike prior work in da… ▽ More Acquiring high-quality training data is essential for current machine learning models. Data markets provide a way to increase the supply of data, particularly in data-scarce domains such as healthcare, by incentivizing potential data sellers to join the market. A major challenge for a data buyer in such a market is selecting the most valuable data points from a data seller. Unlike prior work in data valuation, which assumes centralized data access, we propose a federated approach to the data selection problem that is inspired by linear experimental design. Our proposed data selection method achieves lower prediction error without requiring labeled validation data and can be optimized in a fast and federated procedure. The key insight of our work is that a method that directly estimates the benefit of acquiring data for test set prediction is particularly compatible with a decentralized market setting. △ Less

Submitted 20 March, 2024; originally announced March 2024.

Comments: 26 pages, 20 figures

Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

Authors: Entong Su, Chengzhe Jia, Yuzhe Qin, Wenxuan Zhou, Annabella Macaluso, Binghao Huang, Xiaolong Wang

Abstract: Using tactile sensors for manipulation remains one of the most challenging problems in robotics. At the heart of these challenges is generalization: How can we train a tactile-based policy that can manipulate unseen and diverse objects? In this paper, we propose to perform Reinforcement Learning with only visual tactile sensing inputs on diverse objects in a physical simulator. By training with di… ▽ More Using tactile sensors for manipulation remains one of the most challenging problems in robotics. At the heart of these challenges is generalization: How can we train a tactile-based policy that can manipulate unseen and diverse objects? In this paper, we propose to perform Reinforcement Learning with only visual tactile sensing inputs on diverse objects in a physical simulator. By training with diverse objects in simulation, it enables the policy to generalize to unseen objects. However, leveraging simulation introduces the Sim2Real transfer problem. To mitigate this problem, we study different tactile representations and evaluate how each affects real-robot manipulation results after transfer. We conduct our experiments on diverse real-world objects and show significant improvements over baselines for the pivoting task. Our project page is available at https://tactilerl.github.io/. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Broadband NIR photon upconversion generates NIR persistent luminescence for bioimaging

Authors: Shuting Yang, Bing Qi, Mingzi Sun, Wen**g Dai, Ziyun Miao, Wei Zheng, Bolong Huang, Jie Wang

Abstract: Upconversion persistent luminescence (UCPL) phosphors that can be directly charged by near-infrared (NIR) light have gained considerable attention due to their promising applications ranging from photonics to biomedicine. However, current lanthanide-based UCPL phosphors show small absorption cross-sections and low upconversion charging efficiency. The development of UCPL phosphors faces challenges… ▽ More Upconversion persistent luminescence (UCPL) phosphors that can be directly charged by near-infrared (NIR) light have gained considerable attention due to their promising applications ranging from photonics to biomedicine. However, current lanthanide-based UCPL phosphors show small absorption cross-sections and low upconversion charging efficiency. The development of UCPL phosphors faces challenges of lacking flexible upconversion charging pathways and poor design flexibility. Herein, we discovered a new lattice defect-mediated broadband photon upconversion process and the accompanied NIR-to-NIR UCPL in Cr-doped zinc gallate nanoparticles. The zinc gallate nanoparticles can be directly activated by broadband NIR light in the 700-1000 nm range to produce persistent luminescence at about 700 nm, which is also readily enhanced by rationally tailoring the lattice defects in the phosphors. This proposed UCPL phosphors achieved a signal-to-background ratio of over 200 in bioimaging by efficiently avoiding interference from autofluorescence and light scattering. Our findings reported the lattice defect-mediated photon upconversion for the first time, which significantly expanded the horizons for the flexible design of NIR-to-NIR UCPL phosphors toward broad applications. △ Less

Submitted 14 March, 2024; originally announced March 2024.

LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments

Authors: Maonan Wang, Aoyu Pang, Yuheng Kan, Man-On Pun, Chung Shue Chen, Bo Huang

Abstract: Traffic congestion in metropolitan areas presents a formidable challenge with far-reaching economic, environmental, and societal ramifications. Therefore, effective congestion management is imperative, with traffic signal control (TSC) systems being pivotal in this endeavor. Conventional TSC systems, designed upon rule-based algorithms or reinforcement learning (RL), frequently exhibit deficiencie… ▽ More Traffic congestion in metropolitan areas presents a formidable challenge with far-reaching economic, environmental, and societal ramifications. Therefore, effective congestion management is imperative, with traffic signal control (TSC) systems being pivotal in this endeavor. Conventional TSC systems, designed upon rule-based algorithms or reinforcement learning (RL), frequently exhibit deficiencies in managing the complexities and variabilities of urban traffic flows, constrained by their limited capacity for adaptation to unfamiliar scenarios. In response to these limitations, this work introduces an innovative approach that integrates Large Language Models (LLMs) into TSC, harnessing their advanced reasoning and decision-making faculties. Specifically, a hybrid framework that augments LLMs with a suite of perception and decision-making tools is proposed, facilitating the interrogation of both the static and dynamic traffic information. This design places the LLM at the center of the decision-making process, combining external traffic data with established TSC methods. Moreover, a simulation platform is developed to corroborate the efficacy of the proposed framework. The findings from our simulations attest to the system's adeptness in adjusting to a multiplicity of traffic environments without the need for additional training. Notably, in cases of Sensor Outage (SO), our approach surpasses conventional RL-based systems by reducing the average waiting time by $20.4\%$. This research signifies a notable advance in TSC strategies and paves the way for the integration of LLMs into real-world, dynamic scenarios, highlighting their potential to revolutionize traffic management. The related code is available at https://github.com/Traffic-Alpha/LLM-Assisted-Light. △ Less

Submitted 12 June, 2024; v1 submitted 13 March, 2024; originally announced March 2024.

Comments: 20 pages, 11 figures

Mixed moments of $\rm GL(2)$ and symmetric square $L$-functions

Authors: Bingrong Huang, Liangxun Li

Abstract: In this paper, we prove asymptotic formulas of mixed moments of $\rm GL(2)$ and its symmetric square $L$-functions for both Hecke--Maass cusp forms and holomorphic Hecke eigenforms in short intervals. As an application, we prove quantitative simultaneous non-vanishing of central values of these $L$-functions. In this paper, we prove asymptotic formulas of mixed moments of $\rm GL(2)$ and its symmetric square $L$-functions for both Hecke--Maass cusp forms and holomorphic Hecke eigenforms in short intervals. As an application, we prove quantitative simultaneous non-vanishing of central values of these $L$-functions. △ Less

Submitted 4 April, 2024; v1 submitted 13 March, 2024; originally announced March 2024.

Comments: 25 pages. Comments welcome!

Can Large Language Models Identify Authorship?

Authors: Baixiang Huang, Canyu Chen, Kai Shu

Abstract: The ability to accurately identify authorship is crucial for verifying content authenticity and mitigating misinformation. Large Language Models (LLMs) have demonstrated exceptional capacity for reasoning and problem-solving. However, their potential in authorship analysis, encompassing authorship verification and attribution, remains underexplored. This paper conducts a comprehensive evaluation o… ▽ More The ability to accurately identify authorship is crucial for verifying content authenticity and mitigating misinformation. Large Language Models (LLMs) have demonstrated exceptional capacity for reasoning and problem-solving. However, their potential in authorship analysis, encompassing authorship verification and attribution, remains underexplored. This paper conducts a comprehensive evaluation of LLMs in these critical tasks. Traditional studies have depended on hand-crafted stylistic features, whereas state-of-the-art approaches leverage text embeddings from pre-trained language models. These methods, which typically require fine-tuning on labeled data, often suffer from performance degradation in cross-domain applications and provide limited explainability. This work seeks to address three research questions: (1) Can LLMs perform zero-shot, end-to-end authorship verification effectively? (2) Are LLMs capable of accurately attributing authorship among multiple candidates authors (e.g., 10 and 20)? (3) How can LLMs provide explainability in authorship analysis, particularly through the role of linguistic features? Moreover, we investigate the integration of explicit linguistic features to guide LLMs in their reasoning processes. Our extensive assessment demonstrates LLMs' proficiency in both tasks without the need for domain-specific fine-tuning, providing insights into their decision-making via a detailed analysis of linguistic features. This establishes a new benchmark for future research on LLM-based authorship analysis. The code and data are available at https://github.com/baixianghuang/authorship-llm. △ Less

Submitted 12 March, 2024; originally announced March 2024.

Comments: 10 pages, 7 figures

GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing

Authors: Hao Lu, Xuesong Niu, Jiyao Wang, Yin Wang, Qingyong Hu, Jiaqi Tang, Yuting Zhang, Kaishen Yuan, Bin Huang, Zitong Yu, Dengbo He, Shuiguang Deng, Hao Chen, Yingcong Chen, Shiguang Shan

Abstract: Multimodal large language models (MLLMs) are designed to process and integrate information from multiple sources, such as text, speech, images, and videos. Despite its success in language understanding, it is critical to evaluate the performance of downstream tasks for better human-centric applications. This paper assesses the application of MLLMs with 5 crucial abilities for affective computing,… ▽ More Multimodal large language models (MLLMs) are designed to process and integrate information from multiple sources, such as text, speech, images, and videos. Despite its success in language understanding, it is critical to evaluate the performance of downstream tasks for better human-centric applications. This paper assesses the application of MLLMs with 5 crucial abilities for affective computing, spanning from visual affective tasks and reasoning tasks. The results show that \gpt has high accuracy in facial action unit recognition and micro-expression detection while its general facial expression recognition performance is not accurate. We also highlight the challenges of achieving fine-grained micro-expression recognition and the potential for further study and demonstrate the versatility and potential of \gpt for handling advanced tasks in emotion recognition and related fields by integrating with task-related agents for more complex tasks, such as heart rate estimation through signal processing. In conclusion, this paper provides valuable insights into the potential applications and challenges of MLLMs in human-centric computing. Our interesting examples are at https://github.com/EnVision-Research/GPT4Affectivity. △ Less

Submitted 10 April, 2024; v1 submitted 9 March, 2024; originally announced March 2024.

Blockchain for Finance: A Survey

Authors: Hanjie Wu, Qian Yao, Zhenguang Liu, Butian Huang, Yuan Zhuang, Huayun Tang, Erwu Liu

Abstract: As an innovative technology for enhancing authenticity, security, and risk management, blockchain is being widely adopted in trade and finance systems. The unique capabilities of blockchain, such as immutability and transparency, enable new business models of distributed data storage, point-to-point transactions, and decentralized autonomous organizations. In this paper, we focus on blockchain-bas… ▽ More As an innovative technology for enhancing authenticity, security, and risk management, blockchain is being widely adopted in trade and finance systems. The unique capabilities of blockchain, such as immutability and transparency, enable new business models of distributed data storage, point-to-point transactions, and decentralized autonomous organizations. In this paper, we focus on blockchain-based securities trading, in which blockchain technology plays a vital role in financial services as it ultimately lifts trust and frees the need for third-party verification by using consensus-based verification. We investigate the 12 most popular blockchain platforms and elaborate on 6 platforms that are related to finance, seeking to provide a panorama of securities trading practices. Meanwhile, this survey provides a comprehensive summary of blockchain-based securities trading applications. We gather numerous practical applications of blockchain-based securities trading and categorize them into four distinct categories. For each category, we introduce a typical example and explain how blockchain contributes to solving the key problems faced by FinTech companies and researchers. Finally, we provide interesting observations ranging from mainstream blockchain-based financial institutions to security issues of decentralized finance applications, aiming to picture the current blockchain ecosystem in finance. △ Less

Submitted 27 February, 2024; originally announced February 2024.

RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation

Authors: Hanxiao Jiang, Binghao Huang, Ruihai Wu, Zhuoran Li, Shubham Garg, Hooshang Nayyeri, Shenlong Wang, Yunzhu Li

Abstract: Robots need to explore their surroundings to adapt to and tackle tasks in unknown environments. Prior work has proposed building scene graphs of the environment but typically assumes that the environment is static, omitting regions that require active interactions. This severely limits their ability to handle more complex tasks in household and office environments: before setting up a table, robot… ▽ More Robots need to explore their surroundings to adapt to and tackle tasks in unknown environments. Prior work has proposed building scene graphs of the environment but typically assumes that the environment is static, omitting regions that require active interactions. This severely limits their ability to handle more complex tasks in household and office environments: before setting up a table, robots must explore drawers and cabinets to locate all utensils and condiments. In this work, we introduce the novel task of interactive scene exploration, wherein robots autonomously explore environments and produce an action-conditioned scene graph (ACSG) that captures the structure of the underlying environment. The ACSG accounts for both low-level information, such as geometry and semantics, and high-level information, such as the action-conditioned relationships between different entities in the scene. To this end, we present the Robotic Exploration (RoboEXP) system, which incorporates the Large Multimodal Model (LMM) and an explicit memory design to enhance our system's capabilities. The robot reasons about what and how to explore an object, accumulating new information through the interaction process and incrementally constructing the ACSG. We apply our system across various real-world settings in a zero-shot manner, demonstrating its effectiveness in exploring and modeling environments it has never seen before. Leveraging the constructed ACSG, we illustrate the effectiveness and efficiency of our RoboEXP system in facilitating a wide range of real-world manipulation tasks involving rigid, articulated objects, nested objects like Matryoshka dolls, and deformable objects like cloth. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Comments: Project Page: https://jianghanxiao.github.io/roboexp-web/

Label-efficient Multi-organ Segmentation Method with Diffusion Model

Authors: Yongzhi Huang, **xin Zhu, Haseeb Hassan, Liyilei Su, **gyu Li, Binding Huang

Abstract: Accurate segmentation of multiple organs in Computed Tomography (CT) images plays a vital role in computer-aided diagnosis systems. Various supervised-learning approaches have been proposed recently. However, these methods heavily depend on a large amount of high-quality labeled data, which is expensive to obtain in practice. In this study, we present a label-efficient learning approach using a pr… ▽ More Accurate segmentation of multiple organs in Computed Tomography (CT) images plays a vital role in computer-aided diagnosis systems. Various supervised-learning approaches have been proposed recently. However, these methods heavily depend on a large amount of high-quality labeled data, which is expensive to obtain in practice. In this study, we present a label-efficient learning approach using a pre-trained diffusion model for multi-organ segmentation tasks in CT images. First, a denoising diffusion model was trained using unlabeled CT data, generating additional two-dimensional (2D) CT images. Then the pre-trained denoising diffusion network was transferred to the downstream multi-organ segmentation task, effectively creating a semi-supervised learning model that requires only a small amount of labeled data. Furthermore, linear classification and fine-tuning decoder strategies were employed to enhance the network's segmentation performance. Our generative model at 256x256 resolution achieves impressive performance in terms of Fréchet inception distance, spatial Fréchet inception distance, and F1-score, with values of 11.32, 46.93, and 73.1\%, respectively. These results affirm the diffusion model's ability to generate diverse and realistic 2D CT images. Additionally, our method achieves competitive multi-organ segmentation performance compared to state-of-the-art methods on the FLARE 2022 dataset, particularly in limited labeled data scenarios. Remarkably, even with only 1\% and 10\% labeled data, our method achieves Dice similarity coefficients (DSCs) of 71.56\% and 78.51\% after fine-tuning, respectively. The method achieves a DSC score of 51.81\% using just four labeled CT scans. These results demonstrate the efficacy of our approach in overcoming the limitations of supervised learning heavily reliant on large-scale labeled data. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Federated Causal Discovery from Heterogeneous Data

Authors: Loka Li, Ignavier Ng, Gongxu Luo, Biwei Huang, Guangyi Chen, Tongliang Liu, Bin Gu, Kun Zhang

Abstract: Conventional causal discovery methods rely on centralized data, which is inconsistent with the decentralized nature of data in many real-world situations. This discrepancy has motivated the development of federated causal discovery (FCD) approaches. However, existing FCD methods may be limited by their potentially restrictive assumptions of identifiable functional causal models or homogeneous data… ▽ More Conventional causal discovery methods rely on centralized data, which is inconsistent with the decentralized nature of data in many real-world situations. This discrepancy has motivated the development of federated causal discovery (FCD) approaches. However, existing FCD methods may be limited by their potentially restrictive assumptions of identifiable functional causal models or homogeneous data distributions, narrowing their applicability in diverse scenarios. In this paper, we propose a novel FCD method attempting to accommodate arbitrary causal models and heterogeneous data. We first utilize a surrogate variable corresponding to the client index to account for the data heterogeneity across different clients. We then develop a federated conditional independence test (FCIT) for causal skeleton discovery and establish a federated independent change principle (FICP) to determine causal directions. These approaches involve constructing summary statistics as a proxy of the raw data to protect data privacy. Owing to the nonparametric properties, FCIT and FICP make no assumption about particular functional forms, thereby facilitating the handling of arbitrary causal models. We conduct extensive experiments on synthetic and real datasets to show the efficacy of our method. The code is available at https://github.com/lokali/FedCDH.git. △ Less

Submitted 26 February, 2024; v1 submitted 20 February, 2024; originally announced February 2024.

Comments: ICLR 2024

Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning

Authors: Kaan Ozkara, Bruce Huang, Ruida Zhou, Suhas Diggavi

Abstract: Statistical heterogeneity of clients' local data is an important characteristic in federated learning, motivating personalized algorithms tailored to the local data statistics. Though there has been a plethora of algorithms proposed for personalized supervised learning, discovering the structure of local data through personalized unsupervised learning is less explored. We initiate a systematic stu… ▽ More Statistical heterogeneity of clients' local data is an important characteristic in federated learning, motivating personalized algorithms tailored to the local data statistics. Though there has been a plethora of algorithms proposed for personalized supervised learning, discovering the structure of local data through personalized unsupervised learning is less explored. We initiate a systematic study of such personalized unsupervised learning by develo** algorithms based on optimization criteria inspired by a hierarchical Bayesian statistical framework. We develop adaptive algorithms that discover the balance between using limited local data and collaborative information. We do this in the context of two unsupervised learning tasks: personalized dimensionality reduction and personalized diffusion models. We develop convergence analyses for our adaptive algorithms which illustrate the dependence on problem parameters (e.g., heterogeneity, local sample size). We also develop a theoretical framework for personalized diffusion models, which shows the benefits of collaboration even under heterogeneity. We finally evaluate our proposed algorithms using synthetic and real data, demonstrating the effective sample amplification for personalized tasks, induced through collaboration, despite data heterogeneity. △ Less

Submitted 25 February, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

On the magnetic field properties of protostellar envelopes in Orion

Authors: Bo Huang, Josep M. Girart, Ian W. Stephens, Manuel Fernandez-Lopez, Hector G. Arce, John M. Carpenter, Paulo Cortes, Erin G. Cox, Rachel Friesen, Valentin J. M. Le Gouellec, Charles L. H. Hull, Nicole Karnath, Woo** Kwon, Zhi-Yun Li, Leslie W. Looney, Tom Megeath, Philip C. Myers, Nadia M. Murillo, Jaime E. Pineda, Sarah Sadavoy, Alvaro Sanchez-Monge, Patricio Sanhueza, John J. Tobin, Qizhou Zhang, James M. Jackson , et al. (1 additional authors not shown)

Abstract: We present 870 um polarimetric observations toward 61 protostars in the Orion molecular clouds, with ~400 au (1") resolution using the Atacama Large Millimeter/submillimeter Array. We successfully detect dust polarization and outflow emission in 56 protostars, in 16 of them the polarization is likely produced by self-scattering. Self-scattering signatures are seen in several Class 0 sources, sugge… ▽ More We present 870 um polarimetric observations toward 61 protostars in the Orion molecular clouds, with ~400 au (1") resolution using the Atacama Large Millimeter/submillimeter Array. We successfully detect dust polarization and outflow emission in 56 protostars, in 16 of them the polarization is likely produced by self-scattering. Self-scattering signatures are seen in several Class 0 sources, suggesting that grain growth appears to be significant in disks at earlier protostellar phases. For the rest of the protostars, the dust polarization traces the magnetic field, whose morphology can be approximately classified into three categories: standard-hourglass, rotated-hourglass (with its axis perpendicular to outflow), and spiral-like morphology. 40.0% (+-3.0%) of the protostars exhibit a mean magnetic field direction approximately perpendicular to the outflow on several 100--1000 au scales. However, in the remaining sample, this relative orientation appears to be random, probably due to the complex set of morphologies observed. Furthermore, we classify the protostars into three types based on the C17O (3--2) velocity envelope's gradient: perpendicular to outflow, non-perpendicular to outflow, and unresolved gradient (<1.0~km/s/arcsec). In protostars with a velocity gradient perpendicular to outflow, the magnetic field lines are preferentially perpendicular to outflow, most of them exhibit a rotated hourglass morphology, suggesting that the magnetic field has been overwhelmed by gravity and angular momentum. Spiral-like magnetic fields are associated with envelopes having large velocity gradients, indicating that the rotation motions are strong enough to twist the field lines. All of the protostars with a standard-hourglass field morphology show no significant velocity gradient due to the strong magnetic braking. △ Less

Submitted 15 May, 2024; v1 submitted 11 February, 2024; originally announced February 2024.

Comments: 12 pages, 5 figures (16 pages and 7 figures for the supplementary materials) Accepted for publication on ApJ Letters

Journal ref: ApJL, 963:L31, 2024

Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models

Authors: Yuhang Liu, Zhen Zhang, Dong Gong, Biwei Huang, Mingming Gong, Anton van den Hengel, Kun Zhang, Javen Qinfeng Shi

Abstract: Multimodal contrastive representation learning methods have proven successful across a range of domains, partly due to their ability to generate meaningful shared representations of complex phenomena. To enhance the depth of analysis and understanding of these acquired representations, we introduce a unified causal model specifically designed for multimodal data. By examining this model, we show t… ▽ More Multimodal contrastive representation learning methods have proven successful across a range of domains, partly due to their ability to generate meaningful shared representations of complex phenomena. To enhance the depth of analysis and understanding of these acquired representations, we introduce a unified causal model specifically designed for multimodal data. By examining this model, we show that multimodal contrastive representation learning excels at identifying latent coupled variables within the proposed unified model, up to linear or permutation transformations resulting from different assumptions. Our findings illuminate the potential of pre-trained multimodal models, eg, CLIP, in learning disentangled representations through a surprisingly simple yet highly effective tool: linear independent component analysis. Experiments demonstrate the robustness of our findings, even when the assumptions are violated, and validate the effectiveness of the proposed method in learning disentangled representations. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning

Authors: Binbin Huang, Hailiang Zhao, Lingbin Wang, Wenzhuo Qian, Yuyu Yin, Shuiguang Deng

Abstract: In the resource-constrained IoT-edge environment, Split Federated (SplitFed) learning is implemented to enhance training efficiency. This method involves each IoT device dividing its full DNN model at a designated layer into a device-side model and a server-side model, then offloading the latter to the edge server. However, existing research overlooks four critical issues as follows: (1) the heter… ▽ More In the resource-constrained IoT-edge environment, Split Federated (SplitFed) learning is implemented to enhance training efficiency. This method involves each IoT device dividing its full DNN model at a designated layer into a device-side model and a server-side model, then offloading the latter to the edge server. However, existing research overlooks four critical issues as follows: (1) the heterogeneity of IoT devices' resource capacities and the sizes of their local data samples impact training efficiency; (2) the influence of the edge server's computation and network resource allocation on training efficiency; (3) the data leakage risk associated with the offloaded server-side sub-model; (4) the privacy drawbacks of current centralized algorithms. Consequently, proactively identifying the optimal cut layer and server resource requirements for each IoT device to minimize training latency while adhering to data leakage risk rate constraint remains a challenging issue. To address these problems, this paper first formulates the latency and data leakage risk of training DNN models using Split Federated learning. Next, we frame the Split Federated learning problem as a mixed-integer nonlinear programming challenge. To tackle this, we propose a decentralized Proactive Model Offloading and Resource Allocation (DP-MORA) scheme, empowering each IoT device to determine its cut layer and resource requirements based on its local multidimensional training configuration, without knowledge of other devices' configurations. Extensive experiments on two real-world datasets demonstrate that the DP-MORA scheme effectively reduces DNN model training latency, enhances training efficiency, and complies with data leakage risk constraints compared to several baseline algorithms across various experimental settings. △ Less

Submitted 8 February, 2024; originally announced February 2024.

First measurement of the yield of $^8$He isotopes produced in liquid scintillator by cosmic-ray muons at Daya Bay

Authors: Daya Bay Collaboration, F. P. An, W. D. Bai, A. B. Balantekin, M. Bishai, S. Blyth, G. F. Cao, J. Cao, J. F. Chang, Y. Chang, H. S. Chen, H. Y. Chen, S. M. Chen, Y. Chen, Y. X. Chen, Z. Y. Chen, J. Cheng, Y. C. Cheng, Z. K. Cheng, J. J. Cherwinka, M. C. Chu, J. P. Cummings, O. Dalager, F. S. Deng, X. Y. Ding , et al. (177 additional authors not shown)

Abstract: Daya Bay presents the first measurement of cosmogenic $^8$He isotope production in liquid scintillator, using an innovative method for identifying cascade decays of $^8$He and its child isotope, $^8$Li. We also measure the production yield of $^9$Li isotopes using well-established methodology. The results, in units of 10$^{-8}μ^{-1}$g$^{-1}$cm$^{2}$, are 0.307$\pm$0.042, 0.341$\pm$0.040, and 0.546… ▽ More Daya Bay presents the first measurement of cosmogenic $^8$He isotope production in liquid scintillator, using an innovative method for identifying cascade decays of $^8$He and its child isotope, $^8$Li. We also measure the production yield of $^9$Li isotopes using well-established methodology. The results, in units of 10$^{-8}μ^{-1}$g$^{-1}$cm$^{2}$, are 0.307$\pm$0.042, 0.341$\pm$0.040, and 0.546$\pm$0.076 for $^8$He, and 6.73$\pm$0.73, 6.75$\pm$0.70, and 13.74$\pm$0.82 for $^9$Li at average muon energies of 63.9~GeV, 64.7~GeV, and 143.0~GeV, respectively. The measured production rate of $^8$He isotopes is more than an order of magnitude lower than any other measurement of cosmogenic isotope production. It replaces the results of previous attempts to determine the ratio of $^8$He to $^9$Li production that yielded a wide range of limits from 0 to 30\%. The results provide future liquid-scintillator-based experiments with improved ability to predict cosmogenic backgrounds. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Global existence of a classical solution for the isentropic nozzle flow

Authors: Shih-Wei Chou, Bo-Chih Huang, Yun-guang Lu, Naoki Tsuge

Abstract: Our goal in this paper is to prove the global existence of a classical solution for the isentropic nozzle flow. Regarding this problem, there exist some global existence theorems of weak solutions. However, that of classical solutions does not have much attention until now. When we consider the present problem, the main difficulty is to obtain the uniform bound of solutions and their derivatives.… ▽ More Our goal in this paper is to prove the global existence of a classical solution for the isentropic nozzle flow. Regarding this problem, there exist some global existence theorems of weak solutions. However, that of classical solutions does not have much attention until now. When we consider the present problem, the main difficulty is to obtain the uniform bound of solutions and their derivatives. To solve this, we introduce an invariant region depending on the space variable and a functional satisfying the Riccati equation along the characteristic lines. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Selecting Large Language Model to Fine-tune via Rectified Scaling Law

Authors: Haowei Lin, Baizhou Huang, Haotian Ye, Qinyu Chen, Zihao Wang, Sujian Li, Jianzhu Ma, Xiaojun Wan, James Zou, Yitao Liang

Abstract: The ever-growing ecosystem of LLMs has posed a challenge in selecting the most appropriate pre-trained model to fine-tune amidst a sea of options. Given constrained resources, fine-tuning all models and making selections afterward is unrealistic. In this work, we formulate this resource-constrained selection task into predicting fine-tuning performance and illustrate its natural connection with Sc… ▽ More The ever-growing ecosystem of LLMs has posed a challenge in selecting the most appropriate pre-trained model to fine-tune amidst a sea of options. Given constrained resources, fine-tuning all models and making selections afterward is unrealistic. In this work, we formulate this resource-constrained selection task into predicting fine-tuning performance and illustrate its natural connection with Scaling Law. Unlike pre-training, we find that the fine-tuning scaling curve includes not just the well-known "power phase" but also the previously unobserved "pre-power phase". We also explain why existing Scaling Law fails to capture this phase transition phenomenon both theoretically and empirically. To address this, we introduce the concept of "pre-learned data size" into our Rectified Scaling Law, which overcomes theoretical limitations and fits experimental results much better. By leveraging our law, we propose a novel LLM selection algorithm that selects the near-optimal model with hundreds of times less resource consumption, while other methods may provide negatively correlated selection. The project page is available at rectified-scaling-law.github.io. △ Less

Submitted 28 May, 2024; v1 submitted 3 February, 2024; originally announced February 2024.

Journal ref: ICML 2024

Seeing is not always believing: The Space of Harmless Perturbations

Authors: Lu Chen, Shaofeng Li, Benhao Huang, Fan Yang, Zheng Li, Jie Li, Yuan Luo

Abstract: Existing works have extensively studied adversarial examples, which are minimal perturbations that can mislead the output of deep neural networks (DNNs) while remaining imperceptible to humans. However, in this work, we reveal the existence of a harmless perturbation space, in which perturbations drawn from this space, regardless of their magnitudes, leave the network output unchanged when applied… ▽ More Existing works have extensively studied adversarial examples, which are minimal perturbations that can mislead the output of deep neural networks (DNNs) while remaining imperceptible to humans. However, in this work, we reveal the existence of a harmless perturbation space, in which perturbations drawn from this space, regardless of their magnitudes, leave the network output unchanged when applied to inputs. Essentially, the harmless perturbation space emerges from the usage of non-injective functions (linear or non-linear layers) within DNNs, enabling multiple distinct inputs to be mapped to the same output. For linear layers with input dimensions exceeding output dimensions, any linear combination of the orthogonal bases of the nullspace of the parameter consistently yields no change in their output. For non-linear layers, the harmless perturbation space may expand, depending on the properties of the layers and input samples. Inspired by this property of DNNs, we solve for a family of general perturbation spaces that are redundant for the DNN's decision, and can be used to hide sensitive data and serve as a means of model identification. Our work highlights the distinctive robustness of DNNs (i.e., consistency under large magnitude perturbations) in contrast to adversarial examples (vulnerability for small imperceptible noises). △ Less

Submitted 23 May, 2024; v1 submitted 3 February, 2024; originally announced February 2024.

Natural Counterfactuals With Necessary Backtracking

Authors: Guang-Yuan Hao, Jiji Zhang, Biwei Huang, Hao Wang, Kun Zhang

Abstract: Counterfactual reasoning is pivotal in human cognition and especially important for providing explanations and making decisions. While Judea Pearl's influential approach is theoretically elegant, its generation of a counterfactual scenario often requires interventions that are too detached from the real scenarios to be feasible. In response, we propose a framework of natural counterfactuals and a… ▽ More Counterfactual reasoning is pivotal in human cognition and especially important for providing explanations and making decisions. While Judea Pearl's influential approach is theoretically elegant, its generation of a counterfactual scenario often requires interventions that are too detached from the real scenarios to be feasible. In response, we propose a framework of natural counterfactuals and a method for generating counterfactuals that are natural with respect to the actual world's data distribution. Our methodology refines counterfactual reasoning, allowing changes in causally preceding variables to minimize deviations from realistic scenarios. To generate natural counterfactuals, we introduce an innovative optimization framework that permits but controls the extent of backtracking with a naturalness criterion. Empirical experiments indicate the effectiveness of our method. △ Less

Submitted 20 February, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Octet baryon and heavy meson interactions in chiral effective field theory

Authors: Bo-Lin Huang, Bo Wang, Shi-Lin Zhu

Abstract: We calculate the effective potentials of the octet baryon and heavy meson systems using the chiral effective field theory up to the next-to-leading order. We consider the contact terms, one-pseudoscalar-meson and two-pseudoscalar-meson exchange contributions, facilitating a comprehensive analysis of the short-, mid-, and long-range interactions in these systems. The low energy constants (LECs) are… ▽ More We calculate the effective potentials of the octet baryon and heavy meson systems using the chiral effective field theory up to the next-to-leading order. We consider the contact terms, one-pseudoscalar-meson and two-pseudoscalar-meson exchange contributions, facilitating a comprehensive analysis of the short-, mid-, and long-range interactions in these systems. The low energy constants (LECs) are correlated with those of the $\bar{N}N$ interaction using a quark-level Lagrangian approach. We also incorporate the decuplet baryon contributions in the loop diagrams. Our research provides new insights into several near-threshold charmed baryons [e.g., $Λ_{c}(2940)$, $Ξ_{c}(3055)$, and $Ω_{c}(3188)$, etc.] around $3$ GeV from the hadronic molecular perspective. We also identify several molecular states, designated as $Ξ_{c}$, within the mass range of $3100-3500$ MeV. Further measurements of their resonance parameters and decay patterns in experiments will help to discriminate the conventional baryon and hadronic molecule explanations for these near-threshold states. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Comments: 22 pages, 7 figures and 5 tables

Defining and Extracting generalizable interaction primitives from DNNs

Authors: Lu Chen, Siyu Lou, Benhao Huang, Quanshi Zhang

Abstract: Faithfully summarizing the knowledge encoded by a deep neural network (DNN) into a few symbolic primitive patterns without losing much information represents a core challenge in explainable AI. To this end, Ren et al. (2023c) have derived a series of theorems to prove that the inference score of a DNN can be explained as a small set of interactions between input variables. However, the lack of gen… ▽ More Faithfully summarizing the knowledge encoded by a deep neural network (DNN) into a few symbolic primitive patterns without losing much information represents a core challenge in explainable AI. To this end, Ren et al. (2023c) have derived a series of theorems to prove that the inference score of a DNN can be explained as a small set of interactions between input variables. However, the lack of generalization power makes it still hard to consider such interactions as faithful primitive patterns encoded by the DNN. Therefore, given different DNNs trained for the same task, we develop a new method to extract interactions that are shared by these DNNs. Experiments show that the extracted interactions can better reflect common knowledge shared by different DNNs. △ Less

Submitted 29 January, 2024; originally announced January 2024.

Machine learning for industrial sensing and control: A survey and practical perspective

Authors: Nathan P. Lawrence, Seshu Kumar Damarla, Jong Woo Kim, Aditya Tulsyan, Faraz Amjad, Kai Wang, Benoit Chachuat, Jong Min Lee, Biao Huang, R. Bhushan Gopaluni

Abstract: With the rise of deep learning, there has been renewed interest within the process industries to utilize data on large-scale nonlinear sensing and control problems. We identify key statistical and machine learning techniques that have seen practical success in the process industries. To do so, we start with hybrid modeling to provide a methodological framework underlying core application areas: so… ▽ More With the rise of deep learning, there has been renewed interest within the process industries to utilize data on large-scale nonlinear sensing and control problems. We identify key statistical and machine learning techniques that have seen practical success in the process industries. To do so, we start with hybrid modeling to provide a methodological framework underlying core application areas: soft sensing, process optimization, and control. Soft sensing contains a wealth of industrial applications of statistical and machine learning methods. We quantitatively identify research trends, allowing insight into the most successful techniques in practice. We consider two distinct flavors for data-driven optimization and control: hybrid modeling in conjunction with mathematical programming techniques and reinforcement learning. Throughout these application areas, we discuss their respective industrial requirements and challenges. A common challenge is the interpretability and efficiency of purely data-driven methods. This suggests a need to carefully balance deep learning techniques with domain knowledge. As a result, we highlight ways prior knowledge may be integrated into industrial machine learning applications. The treatment of methods, problems, and applications presented here is poised to inform and inspire practitioners and researchers to develop impactful data-driven sensing, optimization, and control solutions in the process industries. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Comments: 48 pages

Journal ref: Control Engineering Practice 2024

A star-based method for precise flux calibration of the Chinese Space Station Telescope (CSST) slitless spectroscopic survey

Authors: Lin Yang, Haibo Yuan, Fuqing Duan, Ruoyi Zhang, Bowen Huang, Kai Xiao, Shuai Xu, **ming Zhang

Abstract: The upcoming Chinese Space Station Telescope (CSST) slitless spectroscopic survey poses a challenge of flux calibration, which requires a large number of flux-standard stars. In this work, we design an uncertainty-aware residual attention network, the UaRA-net, to derive the CSST SEDs with a resolution of R = 200 over the wavelength range of 2500-10000 Åusing LAMOST normalized spectra with a resol… ▽ More The upcoming Chinese Space Station Telescope (CSST) slitless spectroscopic survey poses a challenge of flux calibration, which requires a large number of flux-standard stars. In this work, we design an uncertainty-aware residual attention network, the UaRA-net, to derive the CSST SEDs with a resolution of R = 200 over the wavelength range of 2500-10000 Åusing LAMOST normalized spectra with a resolution of R = 2000 over the wavelength range of 4000-7000 Å. With the special structure and training strategy, the proposed model can not only provide accurate predictions of SEDs but also their corresponding errors. The precision of the predicted SEDs depends on effective temperature (Teff), wavelength, and the LAMOST spectral signal-to-noise ratios (SNRs), particularly in the GU band. For stars with Teff = 6000 K, the typical SED precisions in the GU band are 4.2%, 2.1%, and 1.5% at SNR values of 20, 40, and 80, respectively. As Teff increases to 8000 K, the precision increases to 1.2%, 0.6%, and 0.5%, respectively. The precision is higher at redder wavelengths. In the GI band, the typical SED precisions for stars with Teff = 6000 K increase to 0.3%, 0.1%, and 0.1% at SNR values of 20, 40, and 80, respectively. We further verify our model using the empirical spectra of the MILES and find good performance. The proposed method will open up new possibilities for optimal utilization of slitless spectra of the CSST and other surveys. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Comments: 20 pages, 15 figures, accepted by ApJS

A comprehensive correction of the Gaia DR3 XP spectra

Authors: Bowen Huang, Haibo Yuan, Maosheng Xiang, Yang Huang, Kai Xiao, Shuai Xu, Ruoyi Zhang, Lin Yang, Zexi Niu, Hongrui Gu

Abstract: By combining spectra from the CALSPEC and NGSL, as well as spectroscopic data from the LAMOST Data Release 7 (DR7), we have analyzed and corrected the systematic errors of the Gaia DR3 BP/RP (XP) spectra. The errors depend on the normalized spectral energy distribution (simplified by two independent ``colors'') and $G$ magnitude. Our corrections are applicable in the range of approximately… ▽ More By combining spectra from the CALSPEC and NGSL, as well as spectroscopic data from the LAMOST Data Release 7 (DR7), we have analyzed and corrected the systematic errors of the Gaia DR3 BP/RP (XP) spectra. The errors depend on the normalized spectral energy distribution (simplified by two independent ``colors'') and $G$ magnitude. Our corrections are applicable in the range of approximately $-0.5<BP-RP<2$, $3<G<17.5$ and $E(B-V)<0.8$. To validate our correction, we conduct independent tests by comparing with the MILES and LEMONY spectra. The results demonstrate that the systematic errors of $BP-RP$ and $G$ have been effectively corrected, especially in the near ultraviolet. The consistency between the corrected Gaia XP spectra and the MILES and LEMONY is better than 2 per cent in the wavelength range of $336-400$\,nm and 1 per cent in redder wavelengths. A global absolute calibration is also carried out by comparing the synthetic Gaia photometry from the corrected XP spectra with the corrected Gaia DR3 photometry. Our study opens up new possibilities for using XP spectra in many fields. A Python package is publicly available to do the corrections (https://doi.org/10.12149/101375 or https://github.com/HiromonGON/GaiaXPcorrection). △ Less

Submitted 22 January, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

Comments: 20 pages, 17 figures, ApJS in Press

Integrated Sensing, Communication, and Computing: An Information-oriented Resource Transaction Mechanism

Authors: Ning Chen, Zhipeng Cheng, Xuwei Fan, Zhang Liu, Bangzhen Huang, Jie Yang, Yifeng Zhao, Lianfen Huang

Abstract: Information acquisition from target perception represents the key enabling technology of the Internet of Automatic Vehicles (IoAV), which is essential for the decision-making and control operation of connected automatic vehicles (CAVs). Exploring target information involves multiple operations on data, e.g., wireless sensing (for data acquisition), communication (for data transmission), and comput… ▽ More Information acquisition from target perception represents the key enabling technology of the Internet of Automatic Vehicles (IoAV), which is essential for the decision-making and control operation of connected automatic vehicles (CAVs). Exploring target information involves multiple operations on data, e.g., wireless sensing (for data acquisition), communication (for data transmission), and computing (for data analysis), which all rely on the consumption of time-space-frequency-computing (TSFC) multi-domain resources. Due to the coupled resource sharing of sensing, communication, and computing procedures, the resource management of information-oriented IoAV is commonly formulated as a non-convex NP-hard problem. In this article, further combining the integrated sensing and communication (ISAC) and computing, we introduce the integrated sensing, communication, and computing (ISCC), wherein the TSFC resources are decoupled from the specific processes and shared universally among sensing, communication, and computing processes. Furthermore, the information-oriented resource trading platform (IRTP) is established, which transforms the problem of ISCC resource management into a resource-information substitution model. Finally, we embed the employment topology structure in IoAV into neural network architecture, taking advantage of the graph neural network (GNN) and multi-worker reinforcement learning, and propose the dynamic resource management strategy based on the asynchronous advantage GNN (A2GNN) algorithm, which can achieve the convergence both of information gain maximization and resource consumption minimization, realizing efficient information-oriented resource management. △ Less

Submitted 22 January, 2024; originally announced January 2024.

Comments: 8 pages, 4 figures, 2 tables