Showing 1–50 of 70 results for author: Ding, X

C^2RV: Cross-Regional and Cross-View Learning for Sparse-View CBCT Reconstruction

Authors: Yiqun Lin, Jiewen Yang, Hualiang Wang, Xinpeng Ding, Wei Zhao, Xiaomeng Li

Abstract: Cone beam computed tomography (CBCT) is an important imaging technology widely used in medical scenarios, such as diagnosis and preoperative planning. Using fewer projection views to reconstruct CT, also known as sparse-view reconstruction, can reduce ionizing radiation and further benefit interventional radiology. Compared with sparse-view reconstruction for traditional parallel/fan-beam CT, CBCT… ▽ More Cone beam computed tomography (CBCT) is an important imaging technology widely used in medical scenarios, such as diagnosis and preoperative planning. Using fewer projection views to reconstruct CT, also known as sparse-view reconstruction, can reduce ionizing radiation and further benefit interventional radiology. Compared with sparse-view reconstruction for traditional parallel/fan-beam CT, CBCT reconstruction is more challenging due to the increased dimensionality caused by the measurement process based on cone-shaped X-ray beams. As a 2D-to-3D reconstruction problem, although implicit neural representations have been introduced to enable efficient training, only local features are considered and different views are processed equally in previous works, resulting in spatial inconsistency and poor performance on complicated anatomies. To this end, we propose C^2RV by leveraging explicit multi-scale volumetric representations to enable cross-regional learning in the 3D space. Additionally, the scale-view cross-attention module is introduced to adaptively aggregate multi-scale and multi-view features. Extensive experiments demonstrate that our C^2RV achieves consistent and significant improvement over previous state-of-the-art methods on datasets with diverse anatomy. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: Accepted to CVPR 2024

Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

Authors: Zhiyu An, Xianzhong Ding, Wan Du

Abstract: Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redes… ▽ More Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and historical data. Our decision tree-based policies are deterministic, verifiable, interpretable, and more energy-efficient than current MBRL methods. First, we introduce a novel verification criterion for RL agents in HVAC control based on domain knowledge. Second, we develop a policy extraction procedure that produces a verifiable decision tree policy. We found that the high dimensionality of the thermal dynamics model input hinders the efficiency of policy extraction. To tackle the dimensionality challenge, we leverage importance sampling conditioned on historical data distributions, significantly improving policy extraction efficiency. Lastly, we present an offline verification algorithm that guarantees the reliability of a control policy. Extensive experiments show that our method saves 68.4% more energy and increases human comfort gain by 14.8% compared to the state-of-the-art method, in addition to an 1127x reduction in computation overhead. Our code and data are available at https://github.com/ryeii/Veri_HVAC △ Less

Submitted 29 February, 2024; originally announced March 2024.

Comments: Accepted for the 61st Design Automation Conference (DAC)

Hunting imaging biomarkers in pulmonary fibrosis: Benchmarks of the AIIB23 challenge

Authors: Yang Nan, Xiaodan Xing, Shiyi Wang, Zeyu Tang, Federico N Felder, Sheng Zhang, Roberta Eufrasia Ledda, Xiaoliu Ding, Ruiqi Yu, Wei** Liu, Feng Shi, Tianyang Sun, Zehong Cao, Minghui Zhang, Yun Gu, Hanxiao Zhang, Jian Gao, **yu Wang, Wen Tang, Pengxin Yu, Han Kang, Junqiang Chen, Xing Lu, Boyu Zhang, Michail Mamalakis , et al. (16 additional authors not shown)

Abstract: Airway-related quantitative imaging biomarkers are crucial for examination, diagnosis, and prognosis in pulmonary diseases. However, the manual delineation of airway trees remains prohibitively time-consuming. While significant efforts have been made towards enhancing airway modelling, current public-available datasets concentrate on lung diseases with moderate morphological variations. The intric… ▽ More Airway-related quantitative imaging biomarkers are crucial for examination, diagnosis, and prognosis in pulmonary diseases. However, the manual delineation of airway trees remains prohibitively time-consuming. While significant efforts have been made towards enhancing airway modelling, current public-available datasets concentrate on lung diseases with moderate morphological variations. The intricate honeycombing patterns present in the lung tissues of fibrotic lung disease patients exacerbate the challenges, often leading to various prediction errors. To address this issue, the 'Airway-Informed Quantitative CT Imaging Biomarker for Fibrotic Lung Disease 2023' (AIIB23) competition was organized in conjunction with the official 2023 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). The airway structures were meticulously annotated by three experienced radiologists. Competitors were encouraged to develop automatic airway segmentation models with high robustness and generalization abilities, followed by exploring the most correlated QIB of mortality prediction. A training set of 120 high-resolution computerised tomography (HRCT) scans were publicly released with expert annotations and mortality status. The online validation set incorporated 52 HRCT scans from patients with fibrotic lung disease and the offline test set included 140 cases from fibrosis and COVID-19 patients. The results have shown that the capacity of extracting airway trees from patients with fibrotic lung disease could be enhanced by introducing voxel-wise weighted general union loss and continuity loss. In addition to the competitive image biomarkers for prognosis, a strong airway-derived biomarker (Hazard ratio>1.5, p<0.0001) was revealed for survival prognostication compared with existing clinical measurements, clinician assessment and AI-based biomarkers. △ Less

Submitted 16 April, 2024; v1 submitted 21 December, 2023; originally announced December 2023.

Comments: 19 pages

Few-Shot Recognition and Classification Framework for Jamming Signal: A CGAN-Based Fusion CNN Approach

Authors: Xuhui Ding, Yue Zhang, Gaoyang Li, Xiaozheng Gao, Neng Ye, Dusit Niyato, Kai Yang

Abstract: Subject to intricate environmental variables, the precise classification of jamming signals holds paramount significance in the effective implementation of anti-jamming strategies within communication systems. In light of this imperative, we propose an innovative fusion algorithm based on conditional generative adversarial network (CGAN) and convolutional neural network (CNN), which aims to deal w… ▽ More Subject to intricate environmental variables, the precise classification of jamming signals holds paramount significance in the effective implementation of anti-jamming strategies within communication systems. In light of this imperative, we propose an innovative fusion algorithm based on conditional generative adversarial network (CGAN) and convolutional neural network (CNN), which aims to deal with the difficulty in applying deep learning (DL) algorithms due to the instantaneous nature of jamming signals in practical communication systems. Compared with previous methods, our algorithm embeds jamming category labels to constrain the range of generated signals in the frequency domain by using the CGAN model, which simultaneously captures potential label information while learning the distribution of signal data thus achieves an 8% improvement in accuracy even when working with a few-sample dataset. Real-world satellite communication scenarios are simulated by adopting hardware platform, and we validate our algorithm by using the resulting time-domain waveform data. The experimental results indicate that our algorithm still performs extremely well, which demonstrates significant potential for practical application in real-world communication scenarios. △ Less

Submitted 26 June, 2024; v1 submitted 9 November, 2023; originally announced November 2023.

Comments: Required to supplement the experiments in Section VII, enhance the notations in Table I, and make necessary adjustments to Equation 17 to ensure accuracy and completeness

Interference Management by Harnessing Multi-Domain Resources in Spectrum-Sharing Aided Satellite-Ground Integrated Networks

Authors: Xiao** Ding, Yue Lei, Yulong Zou, Gengxin Zhang, Lajos Hanzo

Abstract: A spectrum-sharing satellite-ground integrated network is conceived, consisting of a pair of non-geostationary orbit (NGSO) constellations and multiple terrestrial base stations, which impose the co-frequency interference (CFI) on each other. The CFI may increase upon increasing the number of satellites. To manage the potentially severe interference, we propose to rely on joint multi-domain resour… ▽ More A spectrum-sharing satellite-ground integrated network is conceived, consisting of a pair of non-geostationary orbit (NGSO) constellations and multiple terrestrial base stations, which impose the co-frequency interference (CFI) on each other. The CFI may increase upon increasing the number of satellites. To manage the potentially severe interference, we propose to rely on joint multi-domain resource aided interference management (JMDR-IM). Specifically, the coverage overlap of the constellations considered is analyzed. Then, multi-domain resources - including both the beam-domain and power-domain - are jointly utilized for managing the CFI in an overlap** coverage region. This joint resource utilization is performed by relying on our specifically designed beam-shut-off and switching based beam scheduling, as well as on long short-term memory based joint autoregressive moving average assisted deep Q network aided power scheduling. Moreover, the outage probability (OP) of the proposed JMDR-IM scheme is derived, and the asymptotic analysis of the OP is also provided. Our performance evaluations demonstrate the superiority of the proposed JMDR-IM scheme in terms of its increased throughput and reduced OP. △ Less

Submitted 29 January, 2024; v1 submitted 23 October, 2023; originally announced October 2023.

Comments: Submitted to IEEE Transactions on Vehicular Technology

A study on fault diagnosis in nonlinear dynamic systems with uncertainties

Authors: Steven X. Ding, Linlin Li

Abstract: In this draft, fault diagnosis in nonlinear dynamic systems is addressed. The objective of this work is to establish a framework, in which not only model-based but also data-driven and machine learning based fault diagnosis strategies can be uniformly handled. Instead of the well-established input-output and the associated state space models, stable image and kernel representations are adopted in… ▽ More In this draft, fault diagnosis in nonlinear dynamic systems is addressed. The objective of this work is to establish a framework, in which not only model-based but also data-driven and machine learning based fault diagnosis strategies can be uniformly handled. Instead of the well-established input-output and the associated state space models, stable image and kernel representations are adopted in our work as the basic process model forms. Based on it, the nominal system dynamics can then be modelled as a lower-dimensional manifold embedded in the process data space. To achieve a reliable fault detection as a classification problem, projection technique is a capable tool. For nonlinear dynamic systems, we propose to construct projection systems in the well-established framework of Hamiltonian systems and by means of the normalised image and kernel representations. For nonlinear dynamic systems, process data form a non-Euclidean space. Consequently, the norm-based distance defined in Hilbert space is not suitable to measure the distance from a data vector to the manifold of the nominal dynamics. To deal with this issue, we propose to use a Bregman divergence, a measure of difference between two points in a space, as a solution. Moreover, for our purpose of achieving a performance-oriented fault detection, the Bregman divergences adopted in our work are defined by Hamiltonian functions. This scheme not only enables to realise the performance-oriented fault detection, but also uncovers the information geometric aspect of our work. The last part of our work is devoted to the kernel representation based fault detection and uncertainty estimation that can be equivalently used for fault estimation. It is demonstrated that the projection onto the manifold of uncertainty data, together with the correspondingly defined Bregman divergence, is also capable for fault detection. △ Less

Submitted 26 October, 2023; v1 submitted 6 September, 2023; originally announced September 2023.

Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder

Authors: Yusheng Dai, Hang Chen, Jun Du, Xiaofei Ding, Ning Ding, Feijun Jiang, Chin-Hui Lee

Abstract: In recent research, slight performance improvement is observed from automatic speech recognition systems to audio-visual speech recognition systems in the end-to-end framework with low-quality videos. Unmatching convergence rates and specialized input representations between audio and visual modalities are considered to cause the problem. In this paper, we propose two novel techniques to improve a… ▽ More In recent research, slight performance improvement is observed from automatic speech recognition systems to audio-visual speech recognition systems in the end-to-end framework with low-quality videos. Unmatching convergence rates and specialized input representations between audio and visual modalities are considered to cause the problem. In this paper, we propose two novel techniques to improve audio-visual speech recognition (AVSR) under a pre-training and fine-tuning training framework. First, we explore the correlation between lip shapes and syllable-level subword units in Mandarin to establish good frame-level syllable boundaries from lip shapes. This enables accurate alignment of video and audio streams during visual model pre-training and cross-modal fusion. Next, we propose an audio-guided cross-modal fusion encoder (CMFE) neural network to utilize main training parameters for multiple cross-modal attention layers to make full use of modality complementarity. Experiments on the MISP2021-AVSR data set show the effectiveness of the two proposed techniques. Together, using only a relatively small amount of training data, the final system achieves better performances than state-of-the-art systems with more complex front-ends and back-ends. △ Less

Submitted 8 March, 2024; v1 submitted 14 August, 2023; originally announced August 2023.

Comments: 6 pages, 2 figures, published in ICME2023

Leveraging Optical Communication Fiber and AI for Distributed Water Pipe Leak Detection

Authors: Huan Wu, Huan-Feng Duan, Wallace W. L. Lai, Kun Zhu, Xin Cheng, Hao Yin, Bin Zhou, Chun-Cheung Lai, Chao Lu, Xiaoli Ding

Abstract: Detecting leaks in water networks is a costly challenge. This article introduces a practical solution: the integration of optical network with water networks for efficient leak detection. Our approach uses a fiber-optic cable to measure vibrations, enabling accurate leak identification and localization by an intelligent algorithm. We also propose a method to access leak severity for prioritized re… ▽ More Detecting leaks in water networks is a costly challenge. This article introduces a practical solution: the integration of optical network with water networks for efficient leak detection. Our approach uses a fiber-optic cable to measure vibrations, enabling accurate leak identification and localization by an intelligent algorithm. We also propose a method to access leak severity for prioritized repairs. Our solution detects even small leaks with flow rates as low as 0.027 L/s. It offers a cost-effective way to improve leak detection, enhance water management, and increase operational efficiency. △ Less

Submitted 28 July, 2023; originally announced July 2023.

Comments: Accepted

Journal ref: IEEE Communications Magazine, 2023

Learning an Interpretable End-to-End Network for Real-Time Acoustic Beamforming

Authors: Hao Liang, Guanxing Zhou, Xiaotong Tu, Andreas Jakobsson, Xinghao Ding, Yue Huang

Abstract: Recently, many forms of audio industrial applications, such as sound monitoring and source localization, have begun exploiting smart multi-modal devices equipped with a microphone array. Regrettably, model-based methods are often difficult to employ for such devices due to their high computational complexity, as well as the difficulty of appropriately selecting the user-determined parameters. As a… ▽ More Recently, many forms of audio industrial applications, such as sound monitoring and source localization, have begun exploiting smart multi-modal devices equipped with a microphone array. Regrettably, model-based methods are often difficult to employ for such devices due to their high computational complexity, as well as the difficulty of appropriately selecting the user-determined parameters. As an alternative, one may use deep network-based methods, but these are often difficult to generalize, nor can they generate the desired beamforming map directly. In this paper, a computationally efficient acoustic beamforming algorithm is proposed, which may be unrolled to form a model-based deep learning network for real-time imaging, here termed the DAMAS-FISTA-Net. By exploiting the natural structure of an acoustic beamformer, the proposed network inherits the physical knowledge of the acoustic system, and thus learns the underlying physical properties of the propagation. As a result, all the network parameters may be learned end-to-end, guided by a model-based prior using back-propagation. Notably, the proposed network enables an excellent interpretability and the ability of being able to process the raw data directly. Extensive numerical experiments using both simulated and real-world data illustrate the preferable performance of the DAMAS-FISTA-Net as compared to alternative approaches. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Comments: 12 pages, 9 figures

Replay Attack Detection Based on Parity Space Method for Cyber-Physical Systems

Authors: Dong Zhao, Yang Shi, Steven X. Ding, Yueyang Li, Fangzhou Fu

Abstract: The replay attack detection problem is studied from a new perspective based on parity space method in this paper. The proposed detection methods have the ability to distinguish system fault and replay attack, handle both input and output data replay, maintain certain control performance, and can be implemented conveniently and efficiently. First, the replay attack effect on the residual is derived… ▽ More The replay attack detection problem is studied from a new perspective based on parity space method in this paper. The proposed detection methods have the ability to distinguish system fault and replay attack, handle both input and output data replay, maintain certain control performance, and can be implemented conveniently and efficiently. First, the replay attack effect on the residual is derived and analyzed. The residual change induced by replay attack is characterized explicitly and the detection performance analysis based on two different test statistics are given. Second, based on the replay attack effect characterization, targeted passive and active design for detection performance enhancement are proposed. Regarding the passive design, four optimization schemes regarding different cost functions are proposed with optimal parity matrix solutions, and the unified solution to the passive optimization schemes is obtained; the active design is enabled by a marginally stable filter so as to enlarge the replay attack effect on the residual for detection. Simulations and comparison studies are given to show the effectiveness of the proposed methods. △ Less

Submitted 3 June, 2023; originally announced June 2023.

Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems

Authors: Luzhe Huang, Jianing Li, Xiaofu Ding, Yijie Zhang, Hanlong Chen, Aydogan Ozcan

Abstract: Uncertainty estimation is critical for numerous applications of deep neural networks and draws growing attention from researchers. Here, we demonstrate an uncertainty quantification approach for deep neural networks used in inverse problems based on cycle consistency. We build forward-backward cycles using the physical forward model available and a trained deep neural network solving the inverse p… ▽ More Uncertainty estimation is critical for numerous applications of deep neural networks and draws growing attention from researchers. Here, we demonstrate an uncertainty quantification approach for deep neural networks used in inverse problems based on cycle consistency. We build forward-backward cycles using the physical forward model available and a trained deep neural network solving the inverse problem at hand, and accordingly derive uncertainty estimators through regression analysis on the consistency of these forward-backward cycles. We theoretically analyze cycle consistency metrics and derive their relationship with respect to uncertainty, bias, and robustness of the neural network inference. To demonstrate the effectiveness of these cycle consistency-based uncertainty estimators, we classified corrupted and out-of-distribution input image data using some of the widely used image deblurring and super-resolution neural networks as testbeds. The blind testing of our method outperformed other models in identifying unseen input data corruption and distribution shifts. This work provides a simple-to-implement and rapid uncertainty quantification method that can be universally applied to various neural networks used for solving inverse problems. △ Less

Submitted 22 May, 2023; originally announced May 2023.

Comments: 28 Pages, 4 Figures, 1 Table

Journal ref: Intelligent Computing, AAAS (2023)

A Smart Switch Configuration and Reliability Assessment Method for Large-Scale Offshore Wind Farm Electrical Collector System

Authors: Xiaochi Ding, Xinwei Shen, Qiuwei Wu, Liming Wang, Dechang Yang

Abstract: With the development of offshore wind farms (OWFs) in far-offshore and deep-sea areas, each OWF could contain more and more wind turbines and cables, making it imperative to study high-reliability electrical collector system (ECS) for OWF. Enlightened by active distribution network, for OWF, we propose an ECS switch configuration that enables post-fault network recovery, along with a reliability a… ▽ More With the development of offshore wind farms (OWFs) in far-offshore and deep-sea areas, each OWF could contain more and more wind turbines and cables, making it imperative to study high-reliability electrical collector system (ECS) for OWF. Enlightened by active distribution network, for OWF, we propose an ECS switch configuration that enables post-fault network recovery, along with a reliability assessment (RA) method based on optimization models. It can also determine the optimal normal state and network reconfiguration strategies to maximize ECS reliability. Case studies on several OWFs demonstrate that the proposed RA method is more computationally efficient and accurate than the traditional sequential Monte-Carlo simulation method. Moreover, the proposed switch configuration, in conjunction with the network reconfiguration strategy and proper topology, provides significant benefits to ECS reliability. △ Less

Submitted 8 April, 2023; originally announced April 2023.

Comments: 10 pages

Optimizing Irrigation Efficiency using Deep Reinforcement Learning in the Field

Authors: Xianzhong Ding, Wan Du

Abstract: Agricultural irrigation is a significant contributor to freshwater consumption. However, the current irrigation systems used in the field are not efficient. They rely mainly on soil moisture sensors and the experience of growers, but do not account for future soil moisture loss. Predicting soil moisture loss is challenging because it is influenced by numerous factors, including soil texture, weath… ▽ More Agricultural irrigation is a significant contributor to freshwater consumption. However, the current irrigation systems used in the field are not efficient. They rely mainly on soil moisture sensors and the experience of growers, but do not account for future soil moisture loss. Predicting soil moisture loss is challenging because it is influenced by numerous factors, including soil texture, weather conditions, and plant characteristics. This paper proposes a solution to improve irrigation efficiency, which is called DRLIC. DRLIC is a sophisticated irrigation system that uses deep reinforcement learning (DRL) to optimize its performance. The system employs a neural network, known as the DRL control agent, which learns an optimal control policy that considers both the current soil moisture measurement and the future soil moisture loss. We introduce an irrigation reward function that enables our control agent to learn from previous experiences. However, there may be instances where the output of our DRL control agent is unsafe, such as irrigating too much or too little water. To avoid damaging the health of the plants, we implement a safety mechanism that employs a soil moisture predictor to estimate the performance of each action. If the predicted outcome is deemed unsafe, we perform a relatively-conservative action instead. To demonstrate the real-world application of our approach, we developed an irrigation system that comprises sprinklers, sensing and control nodes, and a wireless network. We evaluate the performance of DRLIC by deploying it in a testbed consisting of six almond trees. During a 15-day in-field experiment, we compared the water consumption of DRLIC with a widely-used irrigation scheme. Our results indicate that DRLIC outperformed the traditional irrigation method by achieving a water savings of up to 9.52%. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: 15 pages, 19 figures

Multiple Access Design for Symbiotic Radios: Facilitating Massive IoT Connections with Cellular Networks

Authors: Jun Wang, Xiangyu Ding, Qianqian Zhang, Ying-Chang Liang

Abstract: Symbiotic radio (SR) has emerged as a spectrum- and energy-efficient paradigm to support massive Internet of Things (IoT) connections. Two multiple access schemes are proposed in this paper to facilitate the massive IoT connections using the cellular network based on the SR technique, namely, the simultaneous access (SA) scheme and the selection diversity access (SDA) scheme. In the SA scheme, the… ▽ More Symbiotic radio (SR) has emerged as a spectrum- and energy-efficient paradigm to support massive Internet of Things (IoT) connections. Two multiple access schemes are proposed in this paper to facilitate the massive IoT connections using the cellular network based on the SR technique, namely, the simultaneous access (SA) scheme and the selection diversity access (SDA) scheme. In the SA scheme, the base station (BS) transmits information to the receiver while multiple IoT devices transmit their information simultaneously by passively backscattering the BS signal to the receiver, while in the SDA scheme, only the IoT device with the strongest backscatter link transmits information to the receiver. In both of the schemes, the receiver jointly decodes the information from the BS and the IoT devices. To evaluate the above two schemes, in this paper, we have derived the closed-form expressions of the ergodic rates and the outage probabilities for the cellular and IoT transmissions. Finally, numerical results are provided to verify the theoretical analysis and compare the two proposed multiple access schemes. When the number of IoT devices is small, the SDA scheme is more appealing since it can significantly reduce the computational complexity while achieving equivalent performance to the SA scheme. When the number of IoT devices is large, the SA scheme is preferable since it guarantees a significantly better rate performance and a lower outage probability. △ Less

Submitted 23 March, 2023; originally announced March 2023.

Multi-zone HVAC Control with Model-Based Deep Reinforcement Learning

Authors: Xianzhong Ding, Alberto Cerpa, Wan Du

Abstract: In this paper, we conduct a set of experiments to analyze the limitations of current MBRL-based HVAC control methods, in terms of model uncertainty and controller effectiveness. Using the lessons learned, we develop MB2C, a novel MBRL-based HVAC control system that can achieve high control performance with excellent sample efficiency. MB2C learns the building dynamics by employing an ensemble of e… ▽ More In this paper, we conduct a set of experiments to analyze the limitations of current MBRL-based HVAC control methods, in terms of model uncertainty and controller effectiveness. Using the lessons learned, we develop MB2C, a novel MBRL-based HVAC control system that can achieve high control performance with excellent sample efficiency. MB2C learns the building dynamics by employing an ensemble of environment-conditioned neural networks. It then applies a new control method, Model Predictive Path Integral (MPPI), for HVAC control. It produces candidate action sequences by using an importance sampling weighted algorithm that scales better to high state and action dimensions of multi-zone buildings. We evaluate MB2C using EnergyPlus simulations in a five-zone office building. The results show that MB2C can achieve 8.23% more energy savings compared to the state-of-the-art MBRL solution while maintaining similar thermal comfort. MB2C can reduce the training data set by an order of magnitude (10.52x) while achieving comparable performance to MFRL approaches. △ Less

Submitted 1 February, 2023; originally announced February 2023.

Comments: 13 pages. arXiv admin note: text overlap with arXiv:1708.02596, arXiv:1909.11652 by other authors

Exploring Deep Reinforcement Learning for Holistic Smart Building Control

Authors: Xianzhong Ding, Alberto Cerpa, Wan Du

Abstract: In this paper, we take a holistic approach to deal with the tradeoffs between energy use and comfort in commercial buildings. We developed a system called OCTOPUS, which employs a novel deep reinforcement learning (DRL) framework that uses a data-driven approach to find the optimal control sequences of all building's subsystems, including HVAC, lighting, blind and window systems. The DRL architect… ▽ More In this paper, we take a holistic approach to deal with the tradeoffs between energy use and comfort in commercial buildings. We developed a system called OCTOPUS, which employs a novel deep reinforcement learning (DRL) framework that uses a data-driven approach to find the optimal control sequences of all building's subsystems, including HVAC, lighting, blind and window systems. The DRL architecture includes a novel reward function that allows the framework to explore the tradeoffs between energy use and users' comfort, while at the same time enabling the solution of the high-dimensional control problem due to the interactions of four different building subsystems. In order to cope with OCTOPUS's data training requirements, we argue that calibrated simulations that match the target building operational points are the vehicle to generate enough data to be able to train our DRL framework to find the control solution for the target building. In our work, we trained OCTOPUS with 10-year weather data and a building model that is implemented in the EnergyPlus building simulator, which was calibrated using data from a real production building. Through extensive simulations, we demonstrate that OCTOPUS can achieve 14.26% and 8.1% energy savings compared with the state-of-the-art rule-based method in a LEED Gold Certified building and the latest DRL-based method available in the literature respectively, while maintaining human comfort within a desired range. △ Less

Submitted 26 January, 2023; originally announced January 2023.

Comments: 14 pages. arXiv admin note: text overlap with arXiv:1711.08946 by other authors

A Scale-Arbitrary Image Super-Resolution Network Using Frequency-domain Information

Authors: **g Fang, Yinbo Yu, Zhongyuan Wang, Xin Ding, Ruimin Hu

Abstract: Image super-resolution (SR) is a technique to recover lost high-frequency information in low-resolution (LR) images. Spatial-domain information has been widely exploited to implement image SR, so a new trend is to involve frequency-domain information in SR tasks. Besides, image SR is typically application-oriented and various computer vision tasks call for image arbitrary magnification. Therefore,… ▽ More Image super-resolution (SR) is a technique to recover lost high-frequency information in low-resolution (LR) images. Spatial-domain information has been widely exploited to implement image SR, so a new trend is to involve frequency-domain information in SR tasks. Besides, image SR is typically application-oriented and various computer vision tasks call for image arbitrary magnification. Therefore, in this paper, we study image features in the frequency domain to design a novel scale-arbitrary image SR network. First, we statistically analyze LR-HR image pairs of several datasets under different scale factors and find that the high-frequency spectra of different images under different scale factors suffer from different degrees of degradation, but the valid low-frequency spectra tend to be retained within a certain distribution range. Then, based on this finding, we devise an adaptive scale-aware feature division mechanism using deep reinforcement learning, which can accurately and adaptively divide the frequency spectrum into the low-frequency part to be retained and the high-frequency one to be recovered. Finally, we design a scale-aware feature recovery module to capture and fuse multi-level features for reconstructing the high-frequency spectrum at arbitrary scale factors. Extensive experiments on public datasets show the superiority of our method compared with state-of-the-art methods. △ Less

Submitted 8 December, 2022; originally announced December 2022.

Safety-Critical Optimal Control for Robotic Manipulators in A Cluttered Environment

Authors: Xuda Ding, Han Wang, Yi Ren, Yu Zheng, Cailian Chen, Jian** He

Abstract: Designing safety-critical control for robotic manipulators is challenging, especially in a cluttered environment. First, the actual trajectory of a manipulator might deviate from the planned one due to the complex collision environments and non-trivial dynamics, leading to collision; Second, the feasible space for the manipulator is hard to obtain since the explicit distance functions between coll… ▽ More Designing safety-critical control for robotic manipulators is challenging, especially in a cluttered environment. First, the actual trajectory of a manipulator might deviate from the planned one due to the complex collision environments and non-trivial dynamics, leading to collision; Second, the feasible space for the manipulator is hard to obtain since the explicit distance functions between collision meshes are unknown. By analyzing the relationship between the safe set and the controlled invariant set, this paper proposes a data-driven control barrier function (CBF) construction method, which extracts CBF from distance samples. Specifically, the CBF guarantees the controlled invariant property for considering the system dynamics. The data-driven method samples the distance function and determines the safe set. Then, the CBF is synthesized based on the safe set by a scenario-based sum of square (SOS) program. Unlike most existing linearization based approaches, our method reserves the volume of the feasible space for planning without approximation, which helps find a solution in a cluttered environment. The control law is obtained by solving a CBF-based quadratic program in real time, which works as a safe filter for the desired planning-based controller. Moreover, our method guarantees safety with the proven probabilistic result. Our method is validated on a 7-DOF manipulator in both real and virtual cluttered environments. The experiments show that the manipulator is able to execute tasks where the clearance between obstacles is in millimeters. △ Less

Submitted 10 November, 2022; v1 submitted 9 November, 2022; originally announced November 2022.

Comments: Submitted to IEEE RA-L

Efficient Wideband DoA Estimation with a Robust Iterative Method for Uniform Circular Arrays

Authors: Xiaorui Ding, Wenbo Xu, Yue Wang

Abstract: Direction-of-arrival (DoA) is a critical parameter in wireless channel estimation. With the ever-increasing requirement of high data rate and ubiquitous devices in wireless communication systems, effective wideband DoA estimation is desirable. In this paper, an iterative coherent signal-subspace method including three main steps in each iteration is proposed for wideband two-dimensional (2D) DoA e… ▽ More Direction-of-arrival (DoA) is a critical parameter in wireless channel estimation. With the ever-increasing requirement of high data rate and ubiquitous devices in wireless communication systems, effective wideband DoA estimation is desirable. In this paper, an iterative coherent signal-subspace method including three main steps in each iteration is proposed for wideband two-dimensional (2D) DoA estimation with a uniform circular array. The first step selects partial frequency points for the subsequent focusing process. The second step performs the focusing process, where the angle intervals are designed to generate focusing matrices with robustness, and the signal-subspaces at the selected frequency points are focused into a reference frequency. The third step estimates DoAs with the multiple signal classification (MUSIC) algorithm, where the range of the MUSIC spatial spectrum is constrained by the aforementioned angle intervals. The key parameters of the proposed method in the current iteration are adjusted based on the estimation results in the previous iterations. Besides, the Cramér-Rao bound of the investigated scenario of DoA estimation is derived as a performance benchmark, based on which the guidelines for practical application are provided. The simulation results indicate the proposed method enjoys better estimation performance and preferable efficiency when compared with the benchmark methods. △ Less

Submitted 19 November, 2023; v1 submitted 30 October, 2022; originally announced October 2022.

Weakly Supervised Online Action Detection for Infant General Movements

Authors: Tongyi Luo, Jia Xiao, Chuncao Zhang, Siheng Chen, Yuan Tian, Guangjun Yu, Kang Dang, Xiaowei Ding

Abstract: To make the earlier medical intervention of infants' cerebral palsy (CP), early diagnosis of brain damage is critical. Although general movements assessment(GMA) has shown promising results in early CP detection, it is laborious. Most existing works take videos as input to make fidgety movements(FMs) classification for the GMA automation. Those methods require a complete observation of videos and… ▽ More To make the earlier medical intervention of infants' cerebral palsy (CP), early diagnosis of brain damage is critical. Although general movements assessment(GMA) has shown promising results in early CP detection, it is laborious. Most existing works take videos as input to make fidgety movements(FMs) classification for the GMA automation. Those methods require a complete observation of videos and can not localize video frames containing normal FMs. Therefore we propose a novel approach named WO-GMA to perform FMs localization in the weakly supervised online setting. Infant body keypoints are first extracted as the inputs to WO-GMA. Then WO-GMA performs local spatio-temporal extraction followed by two network branches to generate pseudo clip labels and model online actions. With the clip-level pseudo labels, the action modeling branch learns to detect FMs in an online fashion. Experimental results on a dataset with 757 videos of different infants show that WO-GMA can get state-of-the-art video-level classification and cliplevel detection results. Moreover, only the first 20% duration of the video is needed to get classification results as good as fully observed, implying a significantly shortened FMs diagnosis time. Code is available at: https://github.com/scofiedluo/WO-GMA. △ Less

Submitted 7 August, 2022; originally announced August 2022.

Comments: MICCAI 2022

MSC Class: 68T06 ACM Class: I.2; I.4; J.3

Control theoretically explainable application of autoencoder methods to fault detection in nonlinear dynamic systems

Authors: Linlin Li, Steven X. Ding, Ketian Liang, Zhiwen Chen, Ting Xue

Abstract: This paper is dedicated to control theoretically explainable application of autoencoders to optimal fault detection in nonlinear dynamic systems. Autoencoder-based learning is a standard machine learning method and widely applied for fault (anomaly) detection and classification. In the context of representation learning, the so-called latent (hidden) variable plays an important role towards an opt… ▽ More This paper is dedicated to control theoretically explainable application of autoencoders to optimal fault detection in nonlinear dynamic systems. Autoencoder-based learning is a standard machine learning method and widely applied for fault (anomaly) detection and classification. In the context of representation learning, the so-called latent (hidden) variable plays an important role towards an optimal fault detection. In ideal case, the latent variable should be a minimal sufficient statistic. The existing autoencoder-based fault detection schemes are mainly application-oriented, and few efforts have been devoted to optimal autoencoder-based fault detection and explainable applications. The main objective of our work is to establish a framework for learning autoencoder-based optimal fault detection in nonlinear dynamic systems. To this aim, a process model form for dynamic systems is firstly introduced with the aid of control theory, which also leads to a clear system interpretation of the latent variable. The major efforts are made on the development of a control theoretic solution to the optimal fault detection problem, in which an analog concept to minimal sufficient statistic, the so-called lossless information compression, is introduced and proven for dynamic systems and fault detection specifications. In particular, the existence conditions for such a latent variable are derived, based on which a loss function and further a learning algorithm are developed. This learning algorithm enables optimally training of autoencoders to achieve an optimal fault detection in nonlinear dynamic systems. A case study on three-tank system is given at the end of this paper to illustrate the capability of the proposed autoencoder-based fault detection and to explain the essential role of the latent variable in the proposed fault detection system. △ Less

Submitted 15 May, 2023; v1 submitted 2 August, 2022; originally announced August 2022.

Effective drug combination for Caenorhabditis elegans nematodes discovered by output-driven feedback system control technique

Authors: Xianting Ding, Zach Njus, Taejoon Kong, Wenqiong Su, Chih-Ming Ho, Santosh Pandey

Abstract: Infections from parasitic nematodes (or roundworms) contribute to a significant disease burden and productivity losses for humans and livestock. The limited number of anthelmintics (or antinematode drugs) available today to treat these infections are rapidly losing their efficacy as multidrug resistance in parasites becomes a global health challenge. We propose an engineering approach to discover… ▽ More Infections from parasitic nematodes (or roundworms) contribute to a significant disease burden and productivity losses for humans and livestock. The limited number of anthelmintics (or antinematode drugs) available today to treat these infections are rapidly losing their efficacy as multidrug resistance in parasites becomes a global health challenge. We propose an engineering approach to discover an anthelmintic drug combination that is more potent at killing wild-type Caenorhabditis elegans worms than four individual drugs. In the experiment, freely swimming single worms are enclosed in microfluidic drug environments to assess the centroid velocity and track curvature of worm movements. After analyzing the behavioral data in every iteration, the feedback system control (FSC) scheme is used to predict new drug combinations to test. Through a differential evolutionary search, the winning drug combination is reached that produces minimal centroid velocity and high track curvature, while requiring each drug in less than their EC50 concentrations. The FSC approach is model-less and does not need any information on the drug pharmacology, signaling pathways, or animal biology. Toward combating multidrug resistance, the method presented here is applicable to the discovery of new potent combinations of available anthelmintics on C. elegans, parasitic nematodes, and other small model organisms. △ Less

Submitted 25 May, 2022; originally announced May 2022.

Journal ref: Science Advances 2017

Nonconvex ${L_ {1/2}} $-Regularized Nonlocal Self-similarity Denoiser for Compressive Sensing based CT Reconstruction

Authors: Yunyi Li, Yiqiu Jiang, Hengmin Zhang, Jianxun Liu, Xiangling Ding, Guan Gui

Abstract: Compressive sensing (CS) based computed tomography (CT) image reconstruction aims at reducing the radiation risk through sparse-view projection data. It is usually challenging to achieve satisfying image quality from incomplete projections. Recently, the nonconvex ${L_ {1/2}} $-norm has achieved promising performance in sparse recovery, while the applications on imaging are unsatisfactory due to i… ▽ More Compressive sensing (CS) based computed tomography (CT) image reconstruction aims at reducing the radiation risk through sparse-view projection data. It is usually challenging to achieve satisfying image quality from incomplete projections. Recently, the nonconvex ${L_ {1/2}} $-norm has achieved promising performance in sparse recovery, while the applications on imaging are unsatisfactory due to its nonconvexity. In this paper, we develop a ${L_ {1/2}} $-regularized nonlocal self-similarity (NSS) denoiser for CT reconstruction problem, which integrates low-rank approximation with group sparse coding (GSC) framework. Concretely, we first split the CT reconstruction problem into two subproblems, and then improve the CT image quality furtherly using our ${L_ {1/2}} $-regularized NSS denoiser. Instead of optimizing the nonconvex problem under the perspective of GSC, we particularly reconstruct CT image via low-rank minimization based on two simple yet essential schemes, which build the equivalent relationship between GSC based denoiser and low-rank minimization. Furtherly, the weighted singular value thresholding (WSVT) operator is utilized to optimize the resulting nonconvex ${L_ {1/2}} $ minimization problem. Following this, our proposed denoiser is integrated with the CT reconstruction problem by alternating direction method of multipliers (ADMM) framework. Extensive experimental results on typical clinical CT images have demonstrated that our approach can further achieve better performance than popular approaches. △ Less

Submitted 15 May, 2022; originally announced May 2022.

Comments: Preprint submitted to Journal of The Franklin Institute. Corresponding Author: [email protected], [email protected]

SVR-based Observer Design for Unknown Linear Systems: Complexity and Performance

Authors: Xuda Ding, Han Wang, Jian** He, Cailian Chen, ** Guan

Abstract: In this paper we consider estimating the system parameters and designing stable observer for unknown noisy linear time-invariant (LTI) systems. We propose a Support Vector Regression (SVR) based estimator to provide adjustable asymmetric error interval for estimations. This estimator is capable to trade-off bias-variance of the estimation error by tuning parameter $γ> 0$ in the loss function. This… ▽ More In this paper we consider estimating the system parameters and designing stable observer for unknown noisy linear time-invariant (LTI) systems. We propose a Support Vector Regression (SVR) based estimator to provide adjustable asymmetric error interval for estimations. This estimator is capable to trade-off bias-variance of the estimation error by tuning parameter $γ> 0$ in the loss function. This method enjoys the same sample complexity of $\mathcal{O}(1/\sqrt{N})$ as the Ordinary Least Square (OLS) based methods but achieves a $\mathcal{O}(1/(γ+1))$ smaller variance. Then, a stable observer gain design procedure based on the estimations is proposed. The observation performance bound based on the estimations is evaluated by the mean square observation error, which is shown to be adjustable by tuning the parameter $γ$, thus achieving higher scalability than the OLS methods. The advantages of the estimation error bias-variance trade-off for observer design are also demonstrated through matrix spectrum and observation performance optimality analysis. Extensive simulation validations are conducted to verify the computed estimation error and performance optimality with different $γ$ and noise settings. The variances of the estimation error and the fluctuations in performance are smaller with a properly-designed parameter $γ$ compared with the OLS methods. △ Less

Submitted 14 May, 2022; originally announced May 2022.

Comments: Submitted to AUTOMATICA

Safety-Aware Optimal Control for Motion Planning with Low Computing Complexity

Authors: Xuda Ding, Han Wang, Jian** He, Cailian Chen, Kostas Margellos, Antonis Papachristodoulou

Abstract: The existence of multiple irregular obstacles in the environment introduces nonconvex constraints into the optimization for motion planning, which makes the optimal control problem hard to handle. One efficient approach to address this issue is Successive Convex Approximation (SCA), where the nonconvex problem is convexified and solved successively. However, this approach still faces two main chal… ▽ More The existence of multiple irregular obstacles in the environment introduces nonconvex constraints into the optimization for motion planning, which makes the optimal control problem hard to handle. One efficient approach to address this issue is Successive Convex Approximation (SCA), where the nonconvex problem is convexified and solved successively. However, this approach still faces two main challenges: I) infeasibility, caused by linearisation about infeasible reference points; ii) high computational complexity incurred by multiple constraints, when solving the optimal control problem with a long planning horizon and multiple obstacles. To overcome these challanges, this paper proposes an energy efficient safetyaware control method for motion planning with low computing complexity and address these challenges. Specifically, a control barrier function-based linear quadratic regulator is formulated for the motion planning to guarantee safety and energy efficiency. Then, to avoid infeasibility, Backward Receding SCA (BRSCA) approach with a dynamic constraints-selection rule is proposed. Dynamic programming with primal-dual iteration is designed to decrease computational complexity. It is found that BRSCA is applicable to time-varying control limits. Numerical simulations and hardware experiments vevify the efficiency of BRSCA. Simulations demonstrates that BRSCA has a higher probability of finding feasible solutions, reduces the computation time by about 17.4% and the energy cost by about four times compared to other methods in the literature. △ Less

Submitted 9 November, 2022; v1 submitted 28 April, 2022; originally announced April 2022.

AFSC: Adaptive Fourier Space Compression for Anomaly Detection

Authors: Haote Xu, Yunlong Zhang, Liyan Sun, Chenxin Li, Yue Huang, Xinghao Ding

Abstract: Anomaly Detection (AD) on medical images enables a model to recognize any type of anomaly pattern without lesion-specific supervised learning. Data augmentation based methods construct pseudo-healthy images by "pasting" fake lesions on real healthy ones, and a network is trained to predict healthy images in a supervised manner. The lesion can be found by difference between the unhealthy input and… ▽ More Anomaly Detection (AD) on medical images enables a model to recognize any type of anomaly pattern without lesion-specific supervised learning. Data augmentation based methods construct pseudo-healthy images by "pasting" fake lesions on real healthy ones, and a network is trained to predict healthy images in a supervised manner. The lesion can be found by difference between the unhealthy input and pseudo-healthy output. However, using only manually designed fake lesions fail to approximate to irregular real lesions, hence limiting the model generalization. We assume by exploring the intrinsic data property within images, we can distinguish previously unseen lesions from healthy regions in an unhealthy image. In this study, we propose an Adaptive Fourier Space Compression (AFSC) module to distill healthy feature for AD. The compression of both magnitude and phase in frequency domain addresses the hyper intensity and diverse position of lesions. Experimental results on the BraTS and MS-SEG datasets demonstrate an AFSC baseline is able to produce promising detection results, and an AFSC module can be effectively embedded into existing AD methods. △ Less

Submitted 17 April, 2022; originally announced April 2022.

Comments: 9 pages, 2 figures

Acoustic-Net: A Novel Neural Network for Sound Localization and Quantification

Authors: Guanxing Zhou, Hao Liang, Xinghao Ding, Yue Huang, Xiaotong Tu, Saqlain Abbas

Abstract: Acoustic source localization has been applied in different fields, such as aeronautics and ocean science, generally using multiple microphones array data to reconstruct the source location. However, the model-based beamforming methods fail to achieve the high-resolution of conventional beamforming maps. Deep neural networks are also appropriate to locate the sound source, but in general, these met… ▽ More Acoustic source localization has been applied in different fields, such as aeronautics and ocean science, generally using multiple microphones array data to reconstruct the source location. However, the model-based beamforming methods fail to achieve the high-resolution of conventional beamforming maps. Deep neural networks are also appropriate to locate the sound source, but in general, these methods with complex network structures are hard to be recognized by hardware. In this paper, a novel neural network, termed the Acoustic-Net, is proposed to locate and quantify the sound source simply using the original signals. The experiments demonstrate that the proposed method significantly improves the accuracy of sound source prediction and the computing speed, which may generalize well to real data. The code and trained models are available at https://github.com/JoaquinChou/Acoustic-Net. △ Less

Submitted 31 March, 2022; originally announced March 2022.

Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis

Authors: Yunlong Zhang, Xin Lin, Yihong Zhuang, LiyanSun, Yue Huang, Xinghao Ding, Guisheng Wang, Lin Yang, Yizhou Yu

Abstract: Synthesizing a subject-specific pathology-free image from a pathological image is valuable for algorithm development and clinical practice. In recent years, several approaches based on the Generative Adversarial Network (GAN) have achieved promising results in pseudo-healthy synthesis. However, the discriminator (i.e., a classifier) in the GAN cannot accurately identify lesions and further hampers… ▽ More Synthesizing a subject-specific pathology-free image from a pathological image is valuable for algorithm development and clinical practice. In recent years, several approaches based on the Generative Adversarial Network (GAN) have achieved promising results in pseudo-healthy synthesis. However, the discriminator (i.e., a classifier) in the GAN cannot accurately identify lesions and further hampers from generating admirable pseudo-healthy images. To address this problem, we present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. Then, we apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem existing in medical image segmentation. Furthermore, a reliable metric is proposed by utilizing two attributes of label noise to measure the health of synthetic images. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods. The method achieves better performance than the existing methods with only 30\% of the training data. The effectiveness of the proposed method is also demonstrated on the LiTS and the T1 modality of BraTS. The code and the pre-trained model of this study are publicly available at https://github.com/Au3C2/Generator-Versus-Segmentor. △ Less

Submitted 29 March, 2022; originally announced March 2022.

MDT-Net: Multi-domain Transfer by Perceptual Supervision for Unpaired Images in OCT Scan

Authors: Weinan Song, Gaurav Fotedar, Nima Tajbakhsh, Ziheng Zhou, Lei He, Xiaowei Ding

Abstract: Deep learning models tend to underperform in the presence of domain shifts. Domain transfer has recently emerged as a promising approach wherein images exhibiting a domain shift are transformed into other domains for augmentation or adaptation. However, with the absence of paired and annotated images, models merely learned by adversarial loss and cycle consistency loss could result in poor consist… ▽ More Deep learning models tend to underperform in the presence of domain shifts. Domain transfer has recently emerged as a promising approach wherein images exhibiting a domain shift are transformed into other domains for augmentation or adaptation. However, with the absence of paired and annotated images, models merely learned by adversarial loss and cycle consistency loss could result in poor consistency of anatomy structures during the translation. Additionally, the complexity of learning multi-domain transfer could significantly increase with the number of target domains and source images. In this paper, we propose a multi-domain transfer network, named MDT-Net, to address the limitations above through perceptual supervision. Specifically, our model consists of a single encoder-decoder network and multiple domain-specific transfer modules to disentangle feature representations of the anatomy content and domain variance. Owing to this architecture, the model could significantly reduce the complexity when the translation is conducted among multiple domains. To demonstrate the performance of our method, we evaluate our model qualitatively and quantitatively on RETOUCH, an OCT dataset comprising scans from three different scanner devices (domains). Furthermore, we take the transfer results as additional training data for fluid segmentation to prove the advantage of our model indirectly, i.e., in the task of data adaptation and augmentation. Experimental results show that our method could bring universal improvement in these segmentation tasks, which demonstrates the effectiveness and efficiency of MDT-Net in multi-domain transfer. △ Less

Submitted 25 October, 2022; v1 submitted 12 March, 2022; originally announced March 2022.

An alternative paradigm of fault diagnosis in dynamic systems: orthogonal projection-based methods

Authors: Steven X. Ding, Linlin Li, Tianyu Liu

Abstract: In this paper, we propose a new paradigm of fault diagnosis in dynamic systems as an alternative to the well-established observer-based framework. The basic idea behind this work is to (i) formulate fault detection and isolation as projection of measurement signals onto (system) subspaces in Hilbert space, and (ii) solve the resulting problems by means of projection methods with orthogonal project… ▽ More In this paper, we propose a new paradigm of fault diagnosis in dynamic systems as an alternative to the well-established observer-based framework. The basic idea behind this work is to (i) formulate fault detection and isolation as projection of measurement signals onto (system) subspaces in Hilbert space, and (ii) solve the resulting problems by means of projection methods with orthogonal projection operators and gap metric as major tools. In the new framework, fault diagnosis issues are uniformly addressed both in the model-based and data-driven fashions. Moreover, the design and implementation of the projection-based fault diagnosis systems, from residual generation to threshold setting, can be unifiedly handled. Thanks to the well-defined distance metric for projections in Hilbert subspaces, the projection-based fault diagnosis systems deliver optimal fault detectability. In particular, a new type of residual-driven thresholds is proposed, which significantly increases the fault detectability. In this work, various design schemes are proposed, including a basic projection-based fault detection scheme, fault detection schemes for feedback control systems, fault classification as well as two modified fault detection schemes. As a part of our study, relations to the existing observer-based fault detection systems are investigated, which showcases that, with comparable online computations, the proposed projection-based detection methods offer improved detection performance. △ Less

Submitted 7 May, 2023; v1 submitted 16 February, 2022; originally announced February 2022.

Graph neural network-based fault diagnosis: a review

Authors: Zhiwen Chen, Jiamin Xu, Cesare Alippi, Steven X. Ding, Yuri Shardt, Tao Peng, Chunhua Yang

Abstract: Graph neural network (GNN)-based fault diagnosis (FD) has received increasing attention in recent years, due to the fact that data coming from several application domains can be advantageously represented as graphs. Indeed, this particular representation form has led to superior performance compared to traditional FD approaches. In this review, an easy introduction to GNN, potential applications t… ▽ More Graph neural network (GNN)-based fault diagnosis (FD) has received increasing attention in recent years, due to the fact that data coming from several application domains can be advantageously represented as graphs. Indeed, this particular representation form has led to superior performance compared to traditional FD approaches. In this review, an easy introduction to GNN, potential applications to the field of fault diagnosis, and future perspectives are given. First, the paper reviews neural network-based FD methods by focusing on their data representations, namely, time-series, images, and graphs. Second, basic principles and principal architectures of GNN are introduced, with attention to graph convolutional networks, graph attention networks, graph sample and aggregate, graph auto-encoder, and spatial-temporal graph convolutional networks. Third, the most relevant fault diagnosis methods based on GNN are validated through the detailed experiments, and conclusions are made that the GNN-based methods can achieve good fault diagnosis performance. Finally, discussions and future challenges are provided. △ Less

Submitted 15 November, 2021; originally announced November 2021.

Comments: 17 pages, 18 figures, 10 tables

Self-Verification in Image Denoising

Authors: Huangxing Lin, Yihong Zhuang, Delu Zeng, Yue Huang, Xinghao Ding, John Paisley

Abstract: We devise a new regularization, called self-verification, for image denoising. This regularization is formulated using a deep image prior learned by the network, rather than a traditional predefined prior. Specifically, we treat the output of the network as a ``prior'' that we denoise again after ``re-noising''. The comparison between the again denoised image and its prior can be interpreted as a… ▽ More We devise a new regularization, called self-verification, for image denoising. This regularization is formulated using a deep image prior learned by the network, rather than a traditional predefined prior. Specifically, we treat the output of the network as a ``prior'' that we denoise again after ``re-noising''. The comparison between the again denoised image and its prior can be interpreted as a self-verification of the network's denoising ability. We demonstrate that self-verification encourages the network to capture low-level image statistics needed to restore the image. Based on this self-verification regularization, we further show that the network can learn to denoise even if it has not seen any clean images. This learning strategy is self-supervised, and we refer to it as Self-Verification Image Denoising (SVID). SVID can be seen as a mixture of learning-based methods and traditional model-based denoising methods, in which regularization is adaptively formulated using the output of the network. We show the application of SVID to various denoising tasks using only observed corrupted data. It can achieve the denoising performance close to supervised CNNs. △ Less

Submitted 31 October, 2021; originally announced November 2021.

Semi-Supervised Semantic Segmentation of Vessel Images using Leaking Perturbations

Authors: **yong Hou, Xuejie Ding, Jeremiah D. Deng

Abstract: Semantic segmentation based on deep learning methods can attain appealing accuracy provided large amounts of annotated samples. However, it remains a challenging task when only limited labelled data are available, which is especially common in medical imaging. In this paper, we propose to use Leaking GAN, a GAN-based semi-supervised architecture for retina vessel semantic segmentation. Our key ide… ▽ More Semantic segmentation based on deep learning methods can attain appealing accuracy provided large amounts of annotated samples. However, it remains a challenging task when only limited labelled data are available, which is especially common in medical imaging. In this paper, we propose to use Leaking GAN, a GAN-based semi-supervised architecture for retina vessel semantic segmentation. Our key idea is to pollute the discriminator by leaking information from the generator. This leads to more moderate generations that benefit the training of GAN. As a result, the unlabelled examples can be better utilized to boost the learning of the discriminator, which eventually leads to stronger classification performance. In addition, to overcome the variations in medical images, the mean-teacher mechanism is utilized as an auxiliary regularization of the discriminator. Further, we modify the focal loss to fit it as the consistency objective for mean-teacher regularizer. Extensive experiments demonstrate that the Leaking GAN framework achieves competitive performance compared to the state-of-the-art methods when evaluated on benchmark datasets including DRIVE, STARE and CHASE\_DB1, using as few as 8 labelled images in the semi-supervised setting. It also outperforms existing algorithms on cross-domain segmentation tasks. △ Less

Submitted 22 October, 2021; originally announced October 2021.

Comments: To appear in WACV'22

Bilateral-ViT for Robust Fovea Localization

Authors: Sifan Song, Kang Dang, Qinji Yu, Zilong Wang, Frans Coenen, Jionglong Su, Xiaowei Ding

Abstract: The fovea is an important anatomical landmark of the retina. Detecting the location of the fovea is essential for the analysis of many retinal diseases. However, robust fovea localization remains a challenging problem, as the fovea region often appears fuzzy, and retina diseases may further obscure its appearance. This paper proposes a novel Vision Transformer (ViT) approach that integrates inform… ▽ More The fovea is an important anatomical landmark of the retina. Detecting the location of the fovea is essential for the analysis of many retinal diseases. However, robust fovea localization remains a challenging problem, as the fovea region often appears fuzzy, and retina diseases may further obscure its appearance. This paper proposes a novel Vision Transformer (ViT) approach that integrates information both inside and outside the fovea region to achieve robust fovea localization. Our proposed network, named Bilateral-Vision-Transformer (Bilateral-ViT), consists of two network branches: a transformer-based main network branch for integrating global context across the entire fundus image and a vessel branch for explicitly incorporating the structure of blood vessels. The encoded features from both network branches are subsequently merged with a customized Multi-scale Feature Fusion (MFF) module. Our comprehensive experiments demonstrate that the proposed approach is significantly more robust for diseased images and establishes the new state of the arts using the Messidor and PALM datasets. △ Less

Submitted 3 March, 2022; v1 submitted 19 October, 2021; originally announced October 2021.

Comments: This work has been accepted for oral presentation by ISBI2022

Sneak Attack against Mobile Robotic Networks under Formation Control

Authors: Yushan Li, Jian** He, Xuda Ding, Lin Cai, ** Guan

Abstract: The security of mobile robotic networks (MRNs) has been an active research topic in recent years. This paper demonstrates that the observable interaction process of MRNs under formation control will present increasingly severe threats. Specifically, we find that an external attack robot, who has only partial observation over MRNs while not knowing the system dynamics or access, can learn the inter… ▽ More The security of mobile robotic networks (MRNs) has been an active research topic in recent years. This paper demonstrates that the observable interaction process of MRNs under formation control will present increasingly severe threats. Specifically, we find that an external attack robot, who has only partial observation over MRNs while not knowing the system dynamics or access, can learn the interaction rules from observations and utilize them to replace a target robot, destroying the cooperation performance of MRNs. We call this novel attack as sneak, which endows the attacker with the intelligence of learning knowledge and is hard to be tackled by traditional defense techniques. The key insight is to separately reveal the internal interaction structure within robots and the external interaction mechanism with the environment, from the coupled state evolution influenced by the model-unknown rules and unobservable part of the MRN. To address this issue, we first provide general interaction process modeling and prove the learnability of the interaction rules. Then, with the learned rules, we design an Evaluate-Cut-Restore (ECR) attack strategy considering the partial interaction structure and geometric pattern. We also establish the sufficient conditions for a successful sneak with maximum control impacts over the MRN. Extensive simulations illustrate the feasibility and effectiveness of the proposed attack. △ Less

Submitted 3 June, 2021; originally announced June 2021.

Hierarchical Deep Network with Uncertainty-aware Semi-supervised Learning for Vessel Segmentation

Authors: Chenxin Li, Wenao Ma, Liyan Sun, Xinghao Ding, Yue Huang, Guisheng Wang, Yizhou Yu

Abstract: The analysis of organ vessels is essential for computer-aided diagnosis and surgical planning. But it is not a easy task since the fine-detailed connected regions of organ vessel bring a lot of ambiguity in vessel segmentation and sub-type recognition, especially for the low-contrast capillary regions. Furthermore, recent two-staged approaches would accumulate and even amplify these inaccuracies f… ▽ More The analysis of organ vessels is essential for computer-aided diagnosis and surgical planning. But it is not a easy task since the fine-detailed connected regions of organ vessel bring a lot of ambiguity in vessel segmentation and sub-type recognition, especially for the low-contrast capillary regions. Furthermore, recent two-staged approaches would accumulate and even amplify these inaccuracies from the first-stage whole vessel segmentation into the second-stage sub-type vessel pixel-wise classification. Moreover, the scarcity of manual annotation in organ vessels poses another challenge. In this paper, to address the above issues, we propose a hierarchical deep network where an attention mechanism localizes the low-contrast capillary regions guided by the whole vessels, and enhance the spatial activation in those areas for the sub-type vessels. In addition, we propose an uncertainty-aware semi-supervised training framework to alleviate the annotation-hungry limitation of deep models. The proposed method achieves the state-of-the-art performance in the benchmarks of both retinal artery/vein segmentation in fundus images and liver portal/hepatic vessel segmentation in CT images. △ Less

Submitted 31 May, 2021; originally announced May 2021.

Self-Regression Learning for Blind Hyperspectral Image Fusion Without Label

Authors: Wu Wang, Yue Huang, Xinhao Ding

Abstract: Hyperspectral image fusion (HIF) is critical to a wide range of applications in remote sensing and many computer vision applications. Most traditional HIF methods assume that the observation model is predefined or known. However, in real applications, the observation model involved are often complicated and unknown, which leads to the serious performance drop of many advanced HIF methods. Also, de… ▽ More Hyperspectral image fusion (HIF) is critical to a wide range of applications in remote sensing and many computer vision applications. Most traditional HIF methods assume that the observation model is predefined or known. However, in real applications, the observation model involved are often complicated and unknown, which leads to the serious performance drop of many advanced HIF methods. Also, deep learning methods can achieve outstanding performance, but they generally require a large number of image pairs for model training, which are difficult to obtain in realistic scenarios. Towards these issues, we proposed a self-regression learning method that alternatively reconstructs hyperspectral image (HSI) and estimate the observation model. In particular, we adopt an invertible neural network (INN) for restoring the HSI, and two fully-connected network (FCN) for estimating the observation model. Moreover, \emph{SoftMax} nonlinearity is applied to the FCN for satisfying the non-negative, sparsity and equality constraints. Besides, we proposed a local consistency loss function to constrain the observation model by exploring domain specific knowledge. Finally, we proposed an angular loss function to improve spectral reconstruction accuracy. Extensive experiments on both synthetic and real-world dataset show that our model can outperform the state-of-the-art methods △ Less

Submitted 31 March, 2021; originally announced March 2021.

Unsupervised Anomaly Segmentation using Image-Semantic Cycle Translation

Authors: Chenxin Li, Yunlong Zhang, Jiongcheng Li, Yue Huang, Xinghao Ding

Abstract: The goal of unsupervised anomaly segmentation (UAS) is to detect the pixel-level anomalies unseen during training. It is a promising field in the medical imaging community, e.g, we can use the model trained with only healthy data to segment the lesions of rare diseases. Existing methods are mainly based on Information Bottleneck, whose underlying principle is modeling the distribution of normal an… ▽ More The goal of unsupervised anomaly segmentation (UAS) is to detect the pixel-level anomalies unseen during training. It is a promising field in the medical imaging community, e.g, we can use the model trained with only healthy data to segment the lesions of rare diseases. Existing methods are mainly based on Information Bottleneck, whose underlying principle is modeling the distribution of normal anatomy via learning to compress and recover the healthy data with a low-dimensional manifold, and then detecting lesions as the outlier from this learned distribution. However, this dimensionality reduction inevitably damages the localization information, which is especially essential for pixel-level anomaly detection. In this paper, to alleviate this issue, we introduce the semantic space of healthy anatomy in the process of modeling healthy-data distribution. More precisely, we view the couple of segmentation and synthesis as a special Autoencoder, and propose a novel cycle translation framework with a journey of 'image->semantic->image'. Experimental results on the BraTS and ISLES databases show that the proposed approach achieves significantly superior performance compared to several prior methods and segments the anomalies more accurately. △ Less

Submitted 16 March, 2021; originally announced March 2021.

Robopheus: A Virtual-Physical Interactive Mobile Robotic Testbed

Authors: Xuda Ding, Han Wang, Hongbo Li, Hao Jiang, Jian** He

Abstract: The mobile robotic testbed is an essential and critical support to verify the effectiveness of mobile robotics research. This paper introduces a novel multi-robot testbed, named Robopheus, which exploits the ideas of virtual-physical modeling in digital-twin. Unlike most existing testbeds, the developed Robopheus constructs a bridge that connects the traditional physical hardware and virtual simul… ▽ More The mobile robotic testbed is an essential and critical support to verify the effectiveness of mobile robotics research. This paper introduces a novel multi-robot testbed, named Robopheus, which exploits the ideas of virtual-physical modeling in digital-twin. Unlike most existing testbeds, the developed Robopheus constructs a bridge that connects the traditional physical hardware and virtual simulation testbeds, providing scalable, interactive, and high-fidelity simulations-tests on both sides. Another salient feature of the Robopheus is that it enables a new form to learn the actual models from the physical environment dynamically and is compatible with heterogeneous robot chassis and controllers. In turn, the virtual world's learned models are further leveraged to approximate the robot dynamics online on the physical side. Extensive experiments demonstrate the extraordinary performance of the Robopheus. Significantly, the physical-virtual interaction design increases the trajectory accuracy of a real robot by 300%, compared with that of not using the interaction. △ Less

Submitted 7 March, 2021; originally announced March 2021.

Application of the unified control and detection framework to detecting stealthy integrity cyber-attacks on feedback control systems

Authors: Steven X. Ding, Linlin Li, Dong Zhao, Chris Louen, Tianyu Liu

Abstract: This draft addresses issues of detecting stealthy integrity cyber-attacks on automatic control systems in the unified control and detection framework. A general form of integrity cyber-attacks that cannot be detected using the well-established observer-based technique is first introduced as kernel attacks. The well-known replay, zero dynamics and covert attacks are special forms of the kernel atta… ▽ More This draft addresses issues of detecting stealthy integrity cyber-attacks on automatic control systems in the unified control and detection framework. A general form of integrity cyber-attacks that cannot be detected using the well-established observer-based technique is first introduced as kernel attacks. The well-known replay, zero dynamics and covert attacks are special forms of the kernel attacks. Existence conditions for the kernel attacks are presented. It is demonstrated, in the unified framework of control and detection, that all kernel attacks can be structurally detected when not only the observer-based residual, but also the control signal based residual signals are generated and used for the detection purpose. Based on the analytical results, two schemes for detecting the kernel attacks are then proposed, which allow reliable attack detection without loss of control performance. While the first scheme is similar to the well-established moving target method and auxiliary system aided detection scheme, the second detector is realised with encrypted transmissions of control and monitoring signals in the feedback control system that prevents adversary to gain system knowledge by means of eavesdrop** attacks. Both schemes are illustrated by examples of detecting replay, zero dynamics and covert attacks and an experimental study on a three-tank control system. △ Less

Submitted 4 June, 2021; v1 submitted 27 February, 2021; originally announced March 2021.

Underwater Image Enhancement via Learning Water Type Desensitized Representations

Authors: Zhenqi Fu, Xiaopeng Lin, Wu Wang, Yue Huang, Xinghao Ding

Abstract: We present a novel underwater image enhancement method termed SCNet to improve the image quality meanwhile cope with the degradation diversity caused by the water. SCNet is based on normalization schemes across both spatial and channel dimensions with the key idea of learning water type desensitized features. Specifically, we apply whitening to de-correlate activations across spatial dimensions fo… ▽ More We present a novel underwater image enhancement method termed SCNet to improve the image quality meanwhile cope with the degradation diversity caused by the water. SCNet is based on normalization schemes across both spatial and channel dimensions with the key idea of learning water type desensitized features. Specifically, we apply whitening to de-correlate activations across spatial dimensions for each instance in a mini-batch. We also eliminate channel-wise correlation by standardizing and re-injecting the first two moments of the activations across channels. The normalization schemes of spatial and channel dimensions are performed at each scale of the U-Net to obtain multi-scale representations. With such water type irrelevant encodings, the decoder can easily reconstruct the clean signal and be unaffected by the distortion types. Experimental results on two real-world underwater image datasets show that our approach can successfully enhance images with diverse water types, and achieves competitive performance in visual quality improvement. △ Less

Submitted 14 March, 2022; v1 submitted 1 February, 2021; originally announced February 2021.

Curriculum-based Deep Reinforcement Learning for Quantum Control

Authors: Hailan Ma, Daoyi Dong, Steven X. Ding, Chunlin Chen

Abstract: Deep reinforcement learning has been recognized as an efficient technique to design optimal strategies for different complex systems without prior knowledge of the control landscape. To achieve a fast and precise control for quantum systems, we propose a novel deep reinforcement learning approach by constructing a curriculum consisting of a set of intermediate tasks defined by a fidelity threshold… ▽ More Deep reinforcement learning has been recognized as an efficient technique to design optimal strategies for different complex systems without prior knowledge of the control landscape. To achieve a fast and precise control for quantum systems, we propose a novel deep reinforcement learning approach by constructing a curriculum consisting of a set of intermediate tasks defined by a fidelity threshold. Tasks among a curriculum can be statically determined using empirical knowledge or adaptively generated with the learning process. By transferring knowledge between two successive tasks and sequencing tasks according to their difficulties, the proposed curriculum-based deep reinforcement learning (CDRL) method enables the agent to focus on easy tasks in the early stage, then move onto difficult tasks, and eventually approaches the final task. Numerical simulations on closed quantum systems and open quantum systems demonstrate that the proposed method exhibits improved control performance for quantum systems and also provides an efficient way to identify optimal strategies with fewer control pulses. △ Less

Submitted 2 January, 2021; v1 submitted 30 December, 2020; originally announced December 2020.

Adaptive noise imitation for image denoising

Authors: Huangxing Lin, Yihong Zhuang, Yue Huang, Xinghao Ding, Yizhou Yu, Xiaoqing Liu, John Paisley

Abstract: The effectiveness of existing denoising algorithms typically relies on accurate pre-defined noise statistics or plenty of paired data, which limits their practicality. In this work, we focus on denoising in the more common case where noise statistics and paired data are unavailable. Considering that denoising CNNs require supervision, we develop a new \textbf{adaptive noise imitation (ADANI)} algo… ▽ More The effectiveness of existing denoising algorithms typically relies on accurate pre-defined noise statistics or plenty of paired data, which limits their practicality. In this work, we focus on denoising in the more common case where noise statistics and paired data are unavailable. Considering that denoising CNNs require supervision, we develop a new \textbf{adaptive noise imitation (ADANI)} algorithm that can synthesize noisy data from naturally noisy images. To produce realistic noise, a noise generator takes unpaired noisy/clean images as input, where the noisy image is a guide for noise generation. By imposing explicit constraints on the type, level and gradient of noise, the output noise of ADANI will be similar to the guided noise, while kee** the original clean background of the image. Coupling the noisy data output from ADANI with the corresponding ground-truth, a denoising CNN is then trained in a fully-supervised manner. Experiments show that the noisy data produced by ADANI are visually and statistically similar to real ones so that the denoising CNN in our method is competitive to other networks trained with external paired data. △ Less

Submitted 29 November, 2020; originally announced November 2020.

Perception Matters: Exploring Imperceptible and Transferable Anti-forensics for GAN-generated Fake Face Imagery Detection

Authors: Yongwei Wang, Xin Ding, Li Ding, Rabab Ward, Z. Jane Wang

Abstract: Recently, generative adversarial networks (GANs) can generate photo-realistic fake facial images which are perceptually indistinguishable from real face photos, promoting research on fake face detection. Though fake face forensics can achieve high detection accuracy, their anti-forensic counterparts are less investigated. Here we explore more \textit{imperceptible} and \textit{transferable} anti-f… ▽ More Recently, generative adversarial networks (GANs) can generate photo-realistic fake facial images which are perceptually indistinguishable from real face photos, promoting research on fake face detection. Though fake face forensics can achieve high detection accuracy, their anti-forensic counterparts are less investigated. Here we explore more \textit{imperceptible} and \textit{transferable} anti-forensics for fake face imagery detection based on adversarial attacks. Since facial and background regions are often smooth, even small perturbation could cause noticeable perceptual impairment in fake face images. Therefore it makes existing adversarial attacks ineffective as an anti-forensic method. Our perturbation analysis reveals the intuitive reason of the perceptual degradation issue when directly applying existing attacks. We then propose a novel adversarial attack method, better suitable for image anti-forensics, in the transformed color domain by considering visual perception. Simple yet effective, the proposed method can fool both deep learning and non-deep learning based forensic detectors, achieving higher attack success rate and significantly improved visual quality. Specially, when adversaries consider imperceptibility as a constraint, the proposed anti-forensic method can improve the average attack success rate by around 30\% on fake face images over two baseline attacks. \textit{More imperceptible} and \textit{more transferable}, the proposed method raises new security concerns to fake face imagery detection. We have released our code for public use, and hopefully the proposed method can be further explored in related forensic applications as an anti-forensic benchmark. △ Less

Submitted 29 October, 2020; originally announced October 2020.

Classification Beats Regression: Counting of Cells from Greyscale Microscopic Images based on Annotation-free Training Samples

Authors: Xin Ding, Qiong Zhang, William J. Welch

Abstract: Modern methods often formulate the counting of cells from microscopic images as a regression problem and more or less rely on expensive, manually annotated training images (e.g., dot annotations indicating the centroids of cells or segmentation masks identifying the contours of cells). This work proposes a supervised learning framework based on classification-oriented convolutional neural networks… ▽ More Modern methods often formulate the counting of cells from microscopic images as a regression problem and more or less rely on expensive, manually annotated training images (e.g., dot annotations indicating the centroids of cells or segmentation masks identifying the contours of cells). This work proposes a supervised learning framework based on classification-oriented convolutional neural networks (CNNs) to count cells from greyscale microscopic images without using annotated training images. In this framework, we formulate the cell counting task as an image classification problem, where the cell counts are taken as class labels. This formulation has its limitation when some cell counts in the test stage do not appear in the training data. Moreover, the ordinal relation among cell counts is not utilized. To deal with these limitations, we propose a simple but effective data augmentation (DA) method to synthesize images for the unseen cell counts. We also introduce an ensemble method, which can not only moderate the influence of unseen cell counts but also utilize the ordinal information to improve the prediction accuracy. This framework outperforms many modern cell counting methods and won the data analysis competition (Case Study 1: Counting Cells From Microscopic Images https://ssc.ca/en/case-study/case-study-1-counting-cells-microscopic-images) of the 47th Annual Meeting of the Statistical Society of Canada (SSC). Our code is available at https://github.com/anno2020/CellCount_TinyBBBC005. △ Less

Submitted 29 October, 2020; v1 submitted 28 October, 2020; originally announced October 2020.

Journal ref: The CAAI International Conference on Artificial Intelligence (CICAI 2021)

Multi-Task Neural Networks with Spatial Activation for Retinal Vessel Segmentation and Artery/Vein Classification

Authors: Wenao Ma, Shuang Yu, Kai Ma, Jiexiang Wang, Xinghao Ding, Yefeng Zheng

Abstract: Retinal artery/vein (A/V) classification plays a critical role in the clinical biomarker study of how various systemic and cardiovascular diseases affect the retinal vessels. Conventional methods of automated A/V classification are generally complicated and heavily depend on the accurate vessel segmentation. In this paper, we propose a multi-task deep neural network with spatial activation mechani… ▽ More Retinal artery/vein (A/V) classification plays a critical role in the clinical biomarker study of how various systemic and cardiovascular diseases affect the retinal vessels. Conventional methods of automated A/V classification are generally complicated and heavily depend on the accurate vessel segmentation. In this paper, we propose a multi-task deep neural network with spatial activation mechanism that is able to segment full retinal vessel, artery and vein simultaneously, without the pre-requirement of vessel segmentation. The input module of the network integrates the domain knowledge of widely used retinal preprocessing and vessel enhancement techniques. We specially customize the output block of the network with a spatial activation mechanism, which takes advantage of a relatively easier task of vessel segmentation and exploits it to boost the performance of A/V classification. In addition, deep supervision is introduced to the network to assist the low level layers to extract more semantic information. The proposed network achieves pixel-wise accuracy of 95.70% for vessel segmentation, and A/V classification accuracy of 94.50%, which is the state-of-the-art performance for both tasks on the AV-DRIVE dataset. Furthermore, we have also tested the model performance on INSPIRE-AVR dataset, which achieves a skeletal A/V classification accuracy of 91.6%. △ Less

Submitted 18 July, 2020; originally announced July 2020.

Journal ref: MICCAI 2019

ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting

Authors: Xiaohan Ding, Tianxiang Hao, Jianchao Tan, Ji Liu, Jungong Han, Yuchen Guo, Guiguang Ding

Abstract: We propose ResRep, a novel method for lossless channel pruning (a.k.a. filter pruning), which slims down a CNN by reducing the width (number of output channels) of convolutional layers. Inspired by the neurobiology research about the independence of remembering and forgetting, we propose to re-parameterize a CNN into the remembering parts and forgetting parts, where the former learn to maintain th… ▽ More We propose ResRep, a novel method for lossless channel pruning (a.k.a. filter pruning), which slims down a CNN by reducing the width (number of output channels) of convolutional layers. Inspired by the neurobiology research about the independence of remembering and forgetting, we propose to re-parameterize a CNN into the remembering parts and forgetting parts, where the former learn to maintain the performance and the latter learn to prune. Via training with regular SGD on the former but a novel update rule with penalty gradients on the latter, we realize structured sparsity. Then we equivalently merge the remembering and forgetting parts into the original architecture with narrower layers. In this sense, ResRep can be viewed as a successful application of Structural Re-parameterization. Such a methodology distinguishes ResRep from the traditional learning-based pruning paradigm that applies a penalty on parameters to produce sparsity, which may suppress the parameters essential for the remembering. ResRep slims down a standard ResNet-50 with 76.15% accuracy on ImageNet to a narrower one with only 45% FLOPs and no accuracy drop, which is the first to achieve lossless pruning with such a high compression ratio. The code and models are at https://github.com/DingXiaoH/ResRep. △ Less

Submitted 14 August, 2021; v1 submitted 7 July, 2020; originally announced July 2020.

Comments: ICCV 2021

Cardiac Segmentation on Late Gadolinium Enhancement MRI: A Benchmark Study from Multi-Sequence Cardiac MR Segmentation Challenge

Authors: Xiahai Zhuang, Jiahang Xu, Xinzhe Luo, Chen Chen, Cheng Ouyang, Daniel Rueckert, Victor M. Campello, Karim Lekadir, Sulaiman Vesal, Nishant RaviKumar, Yashu Liu, Gongning Luo, **gkun Chen, Hongwei Li, Buntheng Ly, Maxime Sermesant, Holger Roth, Wentao Zhu, Jiexiang Wang, Xinghao Ding, Xinyue Wang, Sen Yang, Lei Li

Abstract: Accurate computing, analysis and modeling of the ventricles and myocardium from medical images are important, especially in the diagnosis and treatment management for patients suffering from myocardial infarction (MI). Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) provides an important protocol to visualize MI. However, automated segmentation of LGE CMR is still challenging, d… ▽ More Accurate computing, analysis and modeling of the ventricles and myocardium from medical images are important, especially in the diagnosis and treatment management for patients suffering from myocardial infarction (MI). Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) provides an important protocol to visualize MI. However, automated segmentation of LGE CMR is still challenging, due to the indistinguishable boundaries, heterogeneous intensity distribution and complex enhancement patterns of pathological myocardium from LGE CMR. Furthermore, compared with the other sequences LGE CMR images with gold standard labels are particularly limited, which represents another obstacle for develo** novel algorithms for automatic segmentation of LGE CMR. This paper presents the selective results from the Multi-Sequence Cardiac MR (MS-CMR) Segmentation challenge, in conjunction with MICCAI 2019. The challenge offered a data set of paired MS-CMR images, including auxiliary CMR sequences as well as LGE CMR, from 45 patients who underwent cardiomyopathy. It was aimed to develop new algorithms, as well as benchmark existing ones for LGE CMR segmentation and compare them objectively. In addition, the paired MS-CMR images could enable algorithms to combine the complementary information from the other sequences for the segmentation of LGE CMR. Nine representative works were selected for evaluation and comparisons, among which three methods are unsupervised methods and the other six are supervised. The results showed that the average performance of the nine methods was comparable to the inter-observer variations. The success of these methods was mainly attributed to the inclusion of the auxiliary sequences from the MS-CMR images, which provide important label information for the training of deep neural networks. △ Less

Submitted 17 July, 2021; v1 submitted 22 June, 2020; originally announced June 2020.

Comments: 14 pages

Towards Mesh Saliency Detection in 6 Degrees of Freedom

Authors: Xiaoying Ding, Zhenzhong Chen

Abstract: Traditional 3D mesh saliency detection algorithms and corresponding databases were proposed under several constraints such as providing limited viewing directions and not taking the subject's movement into consideration. In this work, a novel 6DoF mesh saliency database is developed which provides both the subject's 6DoF data and eye-movement data. Different from traditional databases, subjects in… ▽ More Traditional 3D mesh saliency detection algorithms and corresponding databases were proposed under several constraints such as providing limited viewing directions and not taking the subject's movement into consideration. In this work, a novel 6DoF mesh saliency database is developed which provides both the subject's 6DoF data and eye-movement data. Different from traditional databases, subjects in the experiment are allowed to move freely to observe 3D meshes in a virtual reality environment. Based on the database, we first analyze the inter-observer variation and the influence of viewing direction towards subject's visual attention, then we provide further investigations about the subject's visual attention bias during observation. Furthermore, we propose a 6DoF mesh saliency detection algorithm based on the uniqueness measure and the bias preference. To evaluate the proposed approach, we also design an evaluation metric accordingly which takes the 6DoF information into consideration, and extend some state-of-the-art 3D saliency detection methods to make comparisons. The experimental results demonstrate the superior performance of our approach for 6DoF mesh saliency detection, in addition to providing benchmarks for the presented 6DoF mesh saliency database. The database and the corresponding algorithms will be made publicly available for research purposes. △ Less

Submitted 22 June, 2020; v1 submitted 26 May, 2020; originally announced May 2020.

Combined Cooling, Heating, and Power System in Blockchain-Enabled Energy Management

Authors: Jianxiong Guo, Xingjian Ding, Weili Wu

Abstract: The combined cooling, heating and power (CCHP) system is a typical distributed, electricity-gas integrated energy scheme in a community. First, it generates electricity by use of gas, and then exploits the waste heat to supply community with heat and cooling. In this paper, we consider a smart city consisting of a number of communities (CCHPs) and an agent of power grid (APG), where CCHPs can sell… ▽ More The combined cooling, heating and power (CCHP) system is a typical distributed, electricity-gas integrated energy scheme in a community. First, it generates electricity by use of gas, and then exploits the waste heat to supply community with heat and cooling. In this paper, we consider a smart city consisting of a number of communities (CCHPs) and an agent of power grid (APG), where CCHPs can sell energy to the APG according to its bid. To study all utilities of entities in such a city from energy trading, a noncooperative Stackelberg game between APG and CCHPs is formulated. Here, the APG gives a bid for buying the energy from CCHPs, then CCHPs respond to the APG with their optimal energy supply that maximizing their utilities according to this bid. We show that the maximum profit to the APG and utilities to the CCHPs can be obtained at the Stackelberg equilibrium, which is guaranteed to exist and unique. Because the complete information about energy supply of each CCHP is unknown to the APG in advance, we propose a distributed algorithm that is able to find the point of equilibrium through a limited number of iterations. Taking privacy protection and transaction security into consideration, we design a blockchain-enabled energy management system. This system is composed of Internet of Energy (IoE) sub-system and blockchain sub-system, where the information interactions as well as energy transactions between APG and CCHPs can be carried out effectively and safely. Finally, security analysis and numerical simulations show the effectiveness and accuracy of our proposed mechanism. △ Less

Submitted 20 March, 2020; originally announced March 2020.

Comments: in IEEE Internet of Things Journal