Showing 1–50 of 54 results for author: Feng, H

CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimation

Authors: Huawei Sun, Hao Feng, Julius Ott, Lorenzo Servadei, Robert Wille

Abstract: Depth estimation is critical in autonomous driving for interpreting 3D scenes accurately. Recently, radar-camera depth estimation has become of sufficient interest due to the robustness and low-cost properties of radar. Thus, this paper introduces a two-stage, end-to-end trainable Confidence-aware Fusion Net (CaFNet) for dense depth estimation, combining RGB imagery with sparse and noisy radar poi… ▽ More Depth estimation is critical in autonomous driving for interpreting 3D scenes accurately. Recently, radar-camera depth estimation has become of sufficient interest due to the robustness and low-cost properties of radar. Thus, this paper introduces a two-stage, end-to-end trainable Confidence-aware Fusion Net (CaFNet) for dense depth estimation, combining RGB imagery with sparse and noisy radar point cloud data. The first stage addresses radar-specific challenges, such as ambiguous elevation and noisy measurements, by predicting a radar confidence map and a preliminary coarse depth map. A novel approach is presented for generating the ground truth for the confidence map, which involves associating each radar point with its corresponding object to identify potential projection surfaces. These maps, together with the initial radar input, are processed by a second encoder. For the final depth estimation, we innovate a confidence-aware gated fusion mechanism to integrate radar and image features effectively, thereby enhancing the reliability of the depth map by filtering out radar noise. Our methodology, evaluated on the nuScenes dataset, demonstrates superior performance, improving upon the current leading model by 3.2% in Mean Absolute Error (MAE) and 2.7% in Root Mean Square Error (RMSE). △ Less

Submitted 30 June, 2024; originally announced July 2024.

Comments: Accepted by IROS 2024

Accurate Patient Alignment without Unnecessary Imaging Dose via Synthesizing Patient-specific 3D CT Images from 2D kV Images

Authors: Yuzhen Ding, Jason M. Holmes, Hongying Feng, Baoxin Li, Lisa A. McGee, Jean-Claude M. Rwigema, Sujay A. Vora, Daniel J. Ma, Robert L. Foote, Samir H. Patel, Wei Liu

Abstract: In radiotherapy, 2D orthogonally projected kV images are used for patient alignment when 3D-on-board imaging(OBI) unavailable. But tumor visibility is constrained due to the projection of patient's anatomy onto a 2D plane, potentially leading to substantial setup errors. In treatment room with 3D-OBI such as cone beam CT(CBCT), the field of view(FOV) of CBCT is limited with unnecessarily high imag… ▽ More In radiotherapy, 2D orthogonally projected kV images are used for patient alignment when 3D-on-board imaging(OBI) unavailable. But tumor visibility is constrained due to the projection of patient's anatomy onto a 2D plane, potentially leading to substantial setup errors. In treatment room with 3D-OBI such as cone beam CT(CBCT), the field of view(FOV) of CBCT is limited with unnecessarily high imaging dose, thus unfavorable for pediatric patients. A solution to this dilemma is to reconstruct 3D CT from kV images obtained at the treatment position. Here, we propose a dual-models framework built with hierarchical ViT blocks. Unlike a proof-of-concept approach, our framework considers kV images as the solo input and can synthesize accurate, full-size 3D CT in real time(within milliseconds). We demonstrate the feasibility of the proposed approach on 10 patients with head and neck (H&N) cancer using image quality(MAE: <45HU), dosimetrical accuracy(Gamma passing rate (2%/2mm/10%)>97%) and patient position uncertainty(shift error: <0.4mm). The proposed framework can generate accurate 3D CT faithfully mirroring real-time patient position, thus significantly improving patient setup accuracy, kee** imaging dose minimum, and maintaining treatment veracity. △ Less

Submitted 1 April, 2024; originally announced May 2024.

Comments: 17 pages, 8 figures and tables

Efficient Navigation of a Robotic Fish Swimming Across the Vortical Flow Field

Authors: Haodong Feng, Dehan Yuan, Jiale Miao, Jie You, Yue Wang, Yi Zhu, Dixia Fan

Abstract: Navigating efficiently across vortical flow fields presents a significant challenge in various robotic applications. The dynamic and unsteady nature of vortical flows often disturbs the control of underwater robots, complicating their operation in hydrodynamic environments. Conventional control methods, which depend on accurate modeling, fail in these settings due to the complexity of fluid-struct… ▽ More Navigating efficiently across vortical flow fields presents a significant challenge in various robotic applications. The dynamic and unsteady nature of vortical flows often disturbs the control of underwater robots, complicating their operation in hydrodynamic environments. Conventional control methods, which depend on accurate modeling, fail in these settings due to the complexity of fluid-structure interactions (FSI) caused by unsteady hydrodynamics. This study proposes a deep reinforcement learning (DRL) algorithm, trained in a data-driven manner, to enable efficient navigation of a robotic fish swimming across vortical flows. Our proposed algorithm incorporates the LSTM architecture and uses several recent consecutive observations as the state to address the issue of partial observation, often due to sensor limitations. We present a numerical study of navigation within a Karman vortex street, created by placing a stationary cylinder in a uniform flow, utilizing the immersed boundary-lattice Boltzmann method (IB-LBM). The aim is to train the robotic fish to discover efficient navigation policies, enabling it to reach a designated target point across the Karman vortex street from various initial positions. After training, the fish demonstrates the ability to rapidly reach the target from different initial positions, showcasing the effectiveness and robustness of our proposed algorithm. Analysis of the results reveals that the robotic fish can leverage velocity gains and pressure differences induced by the vortices to reach the target, underscoring the potential of our proposed algorithm in enhancing navigation in complex hydrodynamic environments. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Enhanced Radar Perception via Multi-Task Learning: Towards Refined Data for Sensor Fusion Applications

Authors: Huawei Sun, Hao Feng, Gianfranco Mauro, Julius Ott, Georg Stettinger, Lorenzo Servadei, Robert Wille

Abstract: Radar and camera fusion yields robustness in perception tasks by leveraging the strength of both sensors. The typical extracted radar point cloud is 2D without height information due to insufficient antennas along the elevation axis, which challenges the network performance. This work introduces a learning-based approach to infer the height of radar points associated with 3D objects. A novel robus… ▽ More Radar and camera fusion yields robustness in perception tasks by leveraging the strength of both sensors. The typical extracted radar point cloud is 2D without height information due to insufficient antennas along the elevation axis, which challenges the network performance. This work introduces a learning-based approach to infer the height of radar points associated with 3D objects. A novel robust regression loss is introduced to address the sparse target challenge. In addition, a multi-task training strategy is employed, emphasizing important features. The average radar absolute height error decreases from 1.69 to 0.25 meters compared to the state-of-the-art height extension method. The estimated target height values are used to preprocess and enrich radar data for downstream perception tasks. Integrating this refined radar information further enhances the performance of existing radar camera fusion models for object detection and depth estimation tasks. △ Less

Submitted 9 April, 2024; originally announced April 2024.

Comments: Accepted by IEEE Intelligent Vehicles Symposium (IV 2024)

Online Signed Sampling of Bandlimited Graph Signals

Authors: Wenwei Liu, Hui Feng, Feng Ji, Bo Hu

Abstract: The theory of sampling and recovery of bandlimited graph signals has been extensively studied. However, in many cases, the observation of a signal is quite coarse. For example, users only provide simple comments such as "like" or "dislike" for a product on an e-commerce platform. This is a particular scenario where only the sign information of a graph signal can be measured. In this paper, we are… ▽ More The theory of sampling and recovery of bandlimited graph signals has been extensively studied. However, in many cases, the observation of a signal is quite coarse. For example, users only provide simple comments such as "like" or "dislike" for a product on an e-commerce platform. This is a particular scenario where only the sign information of a graph signal can be measured. In this paper, we are interested in how to sample based on sign information in an online manner, by which the direction of the original graph signal can be estimated. The online signed sampling problem of a graph signal can be formulated as a Markov decision process in a finite horizon. Unfortunately, it is intractable for large size graphs. We propose a low-complexity greedy signed sampling algorithm (GSS) as well as a stop** criterion. Meanwhile, we prove that the objective function is adaptive monotonic and adaptive submodular, so that the performance is close enough to the global optimum with a lower bound. Finally, we demonstrate the effectiveness of the GSS algorithm by both synthesis and realworld data. △ Less

Submitted 18 February, 2024; v1 submitted 16 February, 2024; originally announced February 2024.

A Fast Power Spectrum Sensing Solution for Generalized Coprime Sampling

Authors: Kaili Jiang, Dechang Wang, Kailun Tian, Hancong Feng, Yuxin Zhao, Junyu Yuan, Bin Tang

Abstract: The growing scarcity of spectrum resources, wideband spectrum sensing is required to process a prohibitive volume of data at a high sampling rate. For some applications, spectrum estimation only requires second-order statistics. In this case, a fast power spectrum sensing solution is proposed based on the generalized coprime sampling. By exploring the sensing vector inherent structure, the autocor… ▽ More The growing scarcity of spectrum resources, wideband spectrum sensing is required to process a prohibitive volume of data at a high sampling rate. For some applications, spectrum estimation only requires second-order statistics. In this case, a fast power spectrum sensing solution is proposed based on the generalized coprime sampling. By exploring the sensing vector inherent structure, the autocorrelation sequence of inputs can be reconstructed from sub-Nyquist samples by only utilizing the parallel Fourier transform and simple multiplication operations. Thus, it takes less time than the state-of-the-art methods while maintaining the same performance, and it achieves higher performance than the existing methods within the same execution time, without the need for pre-estimating the number of inputs. Furthermore, the influence of the model mismatch has only a minor impact on the estimation performance, which allows for more efficient use of the spectrum resource in a distributed swarm scenario. Simulation results demonstrate the low complexity in sampling and computation, making it a more practical solution for real-time and distributed wideband spectrum sensing applications. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Integrated lithium niobate photonic millimeter-wave radar

Authors: Sha Zhu, Yiwen Zhang, Jiaxue Feng, Yongji Wang, Kunpeng Zhai, Hanke Feng, Edwin Yue Bun Pun, Ning Hua Zhu, Cheng Wang

Abstract: Millimeter-wave (mmWave,>30 GHz) radars are the key enabler in the coming 6G era for high-resolution sensing and detection of targets. Photonic radar provides an effective approach to overcome the limitations of electronic radars thanks to the high frequency, broad bandwidth, and excellent reconfigurability of photonic systems. However, conventional photonic radars are mostly realized in tabletop… ▽ More Millimeter-wave (mmWave,>30 GHz) radars are the key enabler in the coming 6G era for high-resolution sensing and detection of targets. Photonic radar provides an effective approach to overcome the limitations of electronic radars thanks to the high frequency, broad bandwidth, and excellent reconfigurability of photonic systems. However, conventional photonic radars are mostly realized in tabletop systems composed of bulky discrete components, whereas the more compact integrated photonic radars are difficult to reach the mmWave bands due to the unsatisfactory bandwidths and signal integrity of the underlining electro-optic modulators. Here, we overcome these challenges and demonstrate a centimeter-resolution integrated photonic radar operating in the mmWave V band (40-50 GHz) based on a 4-inch wafer-scale thin-film lithium niobate (TFLN) technology. The fabricated TFLN mmWave photonic integrated circuit consists of a first electro-optic modulator capable of generating a broadband linear frequency modulated mmWave radar waveform through optical frequency multiplication of a low-frequency input signal, and a second electro-optic modulator responsible for frequency de-chirp of the received reflected echo wave, therefore greatly relieving the bandwidth requirements for the analog-to-digital converter in the receiver. Thanks to the absence of optical and electrical filters in the system, our integrated photonic mmWave radar features continuous on-demand tunability of the center frequency and bandwidth, currently only limited by the bandwidths of electrical amplifiers. We achieve multi-target ranging with a resolution of 1.50 cm and velocity measurement with a resolution of 0.067 m/s. Furthermore, we construct an inverse synthetic aperture radar (ISAR) and successfully demonstrate the imaging of targets with various shapes and postures with a two-dimensional resolution of 1.50 cm * 1.06 cm. △ Less

Submitted 16 November, 2023; originally announced November 2023.

Physics-guided Noise Neural Proxy for Practical Low-light Raw Image Denoising

Authors: Hansen Feng, Lizhi Wang, Yiqi Huang, Yuzhi Wang, Lin Zhu, Hua Huang

Abstract: Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distributio… ▽ More Recently, the mainstream practice for training low-light raw image denoising methods has shifted towards employing synthetic data. Noise modeling, which focuses on characterizing the noise distribution of real-world sensors, profoundly influences the effectiveness and practicality of synthetic data. Currently, physics-based noise modeling struggles to characterize the entire real noise distribution, while learning-based noise modeling impractically depends on paired real data. In this paper, we propose a novel strategy: learning the noise model from dark frames instead of paired real data, to break down the data dependency. Based on this strategy, we introduce an efficient physics-guided noise neural proxy (PNNP) to approximate the real-world sensor noise model. Specifically, we integrate physical priors into neural proxies and introduce three efficient techniques: physics-guided noise decoupling (PND), physics-guided proxy model (PPM), and differentiable distribution loss (DDL). PND decouples the dark frame into different components and handles different levels of noise flexibly, which reduces the complexity of noise modeling. PPM incorporates physical priors to constrain the generated noise, which promotes the accuracy of noise modeling. DDL provides explicit and reliable supervision for noise distribution, which promotes the precision of noise modeling. PNNP exhibits powerful potential in characterizing the real noise distribution. Extensive experiments on public datasets demonstrate superior performance in practical low-light raw image denoising. The code will be available at \url{https://github.com/fenghansen/PNNP}. △ Less

Submitted 22 January, 2024; v1 submitted 13 October, 2023; originally announced October 2023.

Comments: Under Review

Wideband Spectrum Acquisition for UAV Swarm Using the Sparse Coding Fourier Transform

Authors: Kaili Jiang, Kailun Tian, Hancong Feng, Junyu Yuan, Bin Tang

Abstract: As the trend towards small, safe, smart, speedy and swarm development grows, unmanned aerial vehicles (UAVs) are becoming increasingly popular for a wide range of applications. In this letter, the challenge of wideband spectrum acquisition for the UAV swarms is studied by proposing a processing method that features lower power consumption, higher compression rates, and a lower signal-to-noise rati… ▽ More As the trend towards small, safe, smart, speedy and swarm development grows, unmanned aerial vehicles (UAVs) are becoming increasingly popular for a wide range of applications. In this letter, the challenge of wideband spectrum acquisition for the UAV swarms is studied by proposing a processing method that features lower power consumption, higher compression rates, and a lower signal-to-noise ratio. Our system is equipped with multiple UAVs, each with a different sub-sampling rate. That allows for frequency backetization and estimation based on sparse Fourier transform theory. Unlike other techniques, the collisions and iterations caused by non-sparsity environ-ments are considered. We introduce sparse coding Fourier transform to address these issues. The key is to code the entire spectrum and decode it through spectrum correlation in the code. Simulation results show that our proposed method performs well in acquiring both narrowband and wideband signals simultaneously, compared to the other methods. △ Less

Submitted 14 August, 2023; originally announced August 2023.

Distributed UAV Swarm Augmented Wideband Spectrum Sensing Using Nyquist Folding Receiver

Authors: Kaili Jiang, Kailun Tian, Hancong Feng, Yuxin Zhao, Dechang Wang, Sen Cao, Jian Gao, Xuying Zhang, Yanfei Li, Junyu Yuan, Ying Xiong, Bin Tang

Abstract: Distributed unmanned aerial vehicle (UAV) swarms are formed by multiple UAVs with increased portability, higher levels of sensing capabilities, and more powerful autonomy. These features make them attractive for many recent applica-tions, potentially increasing the shortage of spectrum resources. In this paper, wideband spectrum sensing augmented technology is discussed for distributed UAV swarms… ▽ More Distributed unmanned aerial vehicle (UAV) swarms are formed by multiple UAVs with increased portability, higher levels of sensing capabilities, and more powerful autonomy. These features make them attractive for many recent applica-tions, potentially increasing the shortage of spectrum resources. In this paper, wideband spectrum sensing augmented technology is discussed for distributed UAV swarms to improve the utilization of spectrum. However, the sub-Nyquist sampling applied in existing schemes has high hardware complexity, power consumption, and low recovery efficiency for non-strictly sparse conditions. Thus, the Nyquist folding receiver (NYFR) is considered for the distributed UAV swarms, which can theoretically achieve full-band spectrum detection and reception using a single analog-to-digital converter (ADC) at low speed for all circuit components. There is a focus on the sensing model of two multichannel scenarios for the distributed UAV swarms, one with a complete functional receiver for the UAV swarm with RIS, and another with a decentralized UAV swarm equipped with a complete functional receiver for each UAV element. The key issue is to consider whether the application of RIS technology will bring advantages to spectrum sensing and the data fusion problem of decentralized UAV swarms based on the NYFR architecture. Therefore, the property for multiple pulse reconstruction is analyzed through the Gershgorin circle theorem, especially for very short pulses. Further, the block sparse recovery property is analyzed for wide bandwidth signals. The proposed technology can improve the processing capability for multiple signals and wide bandwidth signals while reducing interference from folded noise and subsampled harmonics. Experiment results show augmented spectrum sensing efficiency under non-strictly sparse conditions. △ Less

Submitted 14 August, 2023; originally announced August 2023.

Wideband Power Spectrum Sensing: a Fast Practical Solution for Nyquist Folding Receiver

Authors: Kaili Jiang, Dechang Wang, Kailun Tian, Hancong Feng, Yuxin Zhao, Sen Cao, Jian Gao, Xuying Zhang, Yanfei Li, Junyu Yuan, Ying Xiong, Bin Tang

Abstract: The limited availability of spectrum resources has been growing into a critical problem in wireless communications, remote sensing, and electronic surveillance, etc. To address the high-speed sampling bottleneck of wideband spectrum sensing, a fast and practical solution of power spectrum estimation for Nyquist folding receiver (NYFR) is proposed in this paper. The NYFR architectures is can theore… ▽ More The limited availability of spectrum resources has been growing into a critical problem in wireless communications, remote sensing, and electronic surveillance, etc. To address the high-speed sampling bottleneck of wideband spectrum sensing, a fast and practical solution of power spectrum estimation for Nyquist folding receiver (NYFR) is proposed in this paper. The NYFR architectures is can theoretically achieve the full-band signal sensing with a hundred percent of probability of intercept. But the existing algorithm is difficult to realize in real-time due to its high complexity and complicated calculations. By exploring the sub-sampling principle inherent in NYFR, a computationally efficient method is introduced with compressive covariance sensing. That can be efficient implemented via only the non-uniform fast Fourier transform, fast Fourier transform, and some simple multiplication operations. Meanwhile, the state-of-the-art power spectrum reconstruction model for NYFR of time-domain and frequency-domain is constructed in this paper as a comparison. Furthermore, the computational complexity of the proposed method scales linearly with the Nyquist-rate sampled number of samples and the sparsity of spectrum occupancy. Simulation results and discussion demonstrate that the low complexity in sampling and computation is a more practical solution to meet the real-time wideband spectrum sensing applications. △ Less

Submitted 14 August, 2023; originally announced August 2023.

Segment Anything Model (SAM) for Radiation Oncology

Authors: Lian Zhang, Zhengliang Liu, Lu Zhang, Zihao Wu, Xiaowei Yu, Jason Holmes, Hongying Feng, Haixing Dai, Xiang Li, Quanzheng Li, Dajiang Zhu, Tianming Liu, Wei Liu

Abstract: In this study, we evaluate the performance of the Segment Anything Model (SAM) in clinical radiotherapy. Our results indicate that SAM's 'segment anything' mode can achieve clinically acceptable segmentation results in most organs-at-risk (OARs) with Dice scores higher than 0.7. SAM's 'box prompt' mode further improves the Dice scores by 0.1 to 0.5. Considering the size of the organ and the clarit… ▽ More In this study, we evaluate the performance of the Segment Anything Model (SAM) in clinical radiotherapy. Our results indicate that SAM's 'segment anything' mode can achieve clinically acceptable segmentation results in most organs-at-risk (OARs) with Dice scores higher than 0.7. SAM's 'box prompt' mode further improves the Dice scores by 0.1 to 0.5. Considering the size of the organ and the clarity of its boundary, SAM displays better performance for large organs with clear boundaries but performs worse for smaller organs with unclear boundaries. Given that SAM, a model pre-trained purely on natural images, can handle the delineation of OARs from medical images with clinically acceptable accuracy, these results highlight SAM's robust generalization capabilities with consistent accuracy in automatic segmentation for radiotherapy. In other words, SAM can achieve delineation of different OARs at different sites using a generic automatic segmentation model. SAM's generalization capabilities across different disease sites suggest that it is technically feasible to develop a generic model for automatic segmentation in radiotherapy. △ Less

Submitted 4 July, 2023; v1 submitted 20 June, 2023; originally announced June 2023.

Effects of Tonal Coarticulation and Prosodic Positions on Tonal Contours of Low Rising Tones: In the Case of Xiamen Dialect

Authors: Yiying Hu, Hui Feng, Qinghua Zhao, Aijun Li

Abstract: Few studies have worked on the effects of tonal coarticulation and prosodic positions on the low rising tone in Xiamen Dialect. This study addressed such an issue. To do so, a new method, the Tonal Contour Analysis in Tonal Triangle, was proposed to measure the subtle curvature of the tonal contour. Findings are as follows: (1) The low rising tone in Xiamen Dialect has a tendency towards the falli… ▽ More Few studies have worked on the effects of tonal coarticulation and prosodic positions on the low rising tone in Xiamen Dialect. This study addressed such an issue. To do so, a new method, the Tonal Contour Analysis in Tonal Triangle, was proposed to measure the subtle curvature of the tonal contour. Findings are as follows: (1) The low rising tone in Xiamen Dialect has a tendency towards the falling-rising tone, which is significantly affected by the tonal coarticulation and prosodic positions. (2) The low rising tone presents as a falling-rising tone when preceded by a tone with a high offset, and as a low rising tone when preceded by a tone that ends up low. (3) The curvature of the low rising tone is greatest in the sentence-initial position, and is positively correlated to its own duration. △ Less

Submitted 3 June, 2023; originally announced June 2023.

Comments: To be published in InterSpeech 2023

Active RIS-Assisted mmWave Indoor Signal Enhancement Based on Transparent RIS

Authors: Hao Feng, Yu** Zhao

Abstract: Due to the serious path loss of millimeter-wave (mmWave), the signal sent by the base station is seriously attenuated when it reaches the indoors. Recent studies have proposed a glass-based metasurface that can enhance mmWave indoor signals. The transparent reconfigurable intelligent surface (RIS) focuses on the mmWave signal to a specific location indoors. In this paper, a novel RIS-assisted mmWa… ▽ More Due to the serious path loss of millimeter-wave (mmWave), the signal sent by the base station is seriously attenuated when it reaches the indoors. Recent studies have proposed a glass-based metasurface that can enhance mmWave indoor signals. The transparent reconfigurable intelligent surface (RIS) focuses on the mmWave signal to a specific location indoors. In this paper, a novel RIS-assisted mmWave indoor enhancement scheme is proposed, in which a transparent RIS is deployed on the glass to enhance mmWave indoor signals, and three assisted transmission scenarios, namely passive RIS (PRIS), active RIS (ARIS), and a novel hybrid RIS (HRIS) are proposed. This paper aims to maximize the signal-to-noise ratio (SNR) of the received signal for the three assisted transmission scenarios. The closed-form solution to the maximum SNR is presented in the PRIS and the ARIS-assisted transmission scenarios. Meanwhile, the closed-form solution to the maximum SNR for the HRIS-assisted transmission scenario is presented for given active unit cells. In addition, the performance of the proposed scheme is analyzed under three assisted transmission scenarios. The results indicate that under a specific RIS power budget, the ARIS-assisted transmission scenario achieves the highest data rate and energy efficiency. Also, it requires very few unit cells, thus dramatically reducing the size of the metasurface. △ Less

Submitted 16 May, 2023; originally announced May 2023.

Optical Aberration Correction in Postprocessing using Imaging Simulation

Authors: Shiqi Chen, Huajun Feng, Dexin Pan, Zhihai Xu, Qi Li, Yueting Chen

Abstract: As the popularity of mobile photography continues to grow, considerable effort is being invested in the reconstruction of degraded images. Due to the spatial variation in optical aberrations, which cannot be avoided during the lens design process, recent commercial cameras have shifted some of these correction tasks from optical design to postprocessing systems. However, without engaging with the… ▽ More As the popularity of mobile photography continues to grow, considerable effort is being invested in the reconstruction of degraded images. Due to the spatial variation in optical aberrations, which cannot be avoided during the lens design process, recent commercial cameras have shifted some of these correction tasks from optical design to postprocessing systems. However, without engaging with the optical parameters, these systems only achieve limited correction for aberrations.In this work, we propose a practical method for recovering the degradation caused by optical aberrations. Specifically, we establish an imaging simulation system based on our proposed optical point spread function model. Given the optical parameters of the camera, it generates the imaging results of these specific devices. To perform the restoration, we design a spatial-adaptive network model on synthetic data pairs generated by the imaging simulation system, eliminating the overhead of capturing training data by a large amount of shooting and registration. Moreover, we comprehensively evaluate the proposed method in simulations and experimentally with a customized digital-single-lens-reflex (DSLR) camera lens and HUAWEI HONOR 20, respectively. The experiments demonstrate that our solution successfully removes spatially variant blur and color dispersion. When compared with the state-of-the-art deblur methods, the proposed approach achieves better results with a lower computational overhead. Moreover, the reconstruction technique does not introduce artificial texture and is convenient to transfer to current commercial cameras. Project Page: \url{https://github.com/TanGeeGo/ImagingSimulation}. △ Less

Submitted 9 May, 2023; originally announced May 2023.

Comments: Published in ACM TOG. 15 pages, 13 figures

Journal ref: ACM Trans. Graph. 40, 5, Article 192 (October 2021), 15 pages

Model-Based Monitoring and State Estimation for Digital Twins: The Kalman Filter

Authors: Hao Feng, Cláudio Gomes, Peter Gorm Larsen

Abstract: A digital twin (DT) monitors states of the physical twin (PT) counterpart and provides a number of benefits such as advanced visualizations, fault detection capabilities, and reduced maintenance cost. It is the ability to be able to detect the states inside the DT that enable such benefits. In order to estimate the desired states of a PT, we propose the use of a Kalman Filter (KF). In this tutoria… ▽ More A digital twin (DT) monitors states of the physical twin (PT) counterpart and provides a number of benefits such as advanced visualizations, fault detection capabilities, and reduced maintenance cost. It is the ability to be able to detect the states inside the DT that enable such benefits. In order to estimate the desired states of a PT, we propose the use of a Kalman Filter (KF). In this tutorial, we provide an introduction and detailed derivation of the KF. We demonstrate the use of KF to monitor and anomaly detection through an incubator system. Our experimental result shows that KF successfully can detect the anomaly during monitoring. △ Less

Submitted 29 April, 2023; originally announced May 2023.

Comments: 14 pages

mmWave RIS Phase Shift Feedback Based on Knowledge Base Autoencoder Framework

Authors: Hao Feng, Yuting Xu, Yu** Zhao

Abstract: In reconfigurable intelligent surface (RIS)-assisted wireless communication systems, adjusting the phase shift of RIS unit cells is crucial for improving communication performance. Due to massive RIS unit cells, the number of phase shift parameters fed back from the base station (BS) to the RIS is enormous, which occupies a large number of frequency resources. In this paper, we propose a feedback… ▽ More In reconfigurable intelligent surface (RIS)-assisted wireless communication systems, adjusting the phase shift of RIS unit cells is crucial for improving communication performance. Due to massive RIS unit cells, the number of phase shift parameters fed back from the base station (BS) to the RIS is enormous, which occupies a large number of frequency resources. In this paper, we propose a feedback scheme for millimeter-wave RIS phase shift applying a knowledge base autoencoder framework, in which the learnable knowledge base is shared at the BS and the RIS. The encoder at the BS compresses the RIS phase shift matrix to multiple feature vectors. Then the knowledge base vectors index is obtained by calculating the similarity between feature vectors and knowledge base vectors and transmitted to the RIS. With utilizing the index at the RIS, the corresponding knowledge base vectors are extracted and used as the decoder's inputs to reconstruct the phase shift of the RIS. Simulation results show that the proposed scheme can significantly improve the accuracy of phase shift feedback and impressively reduce the amount of RIS phase shift feedback data. Moreover, the proposed scheme is easy to deploy in actual scenarios due to lower complexity and fewer parameters. △ Less

Submitted 27 April, 2023; originally announced April 2023.

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Authors: Haodong Feng, Yue Wang, Hui Xiang, Zhiyang **, Dixia Fan

Abstract: Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control th… ▽ More Deep reinforcement learning (DRL) for fluidic pinball, three individually rotating cylinders in the uniform flow arranged in an equilaterally triangular configuration, can learn the efficient flow control strategies due to the validity of self-learning and data-driven state estimation for complex fluid dynamic problems. In this work, we present a DRL-based real-time feedback strategy to control the hydrodynamic force on fluidic pinball, i.e., force extremum and tracking, from cylinders' rotation. By adequately designing reward functions and encoding historical observations, and after automatic learning of thousands of iterations, the DRL-based control was shown to make reasonable and valid control decisions in nonparametric control parameter space, which is comparable to and even better than the optimal policy found through lengthy brute-force searching. Subsequently, one of these results was analyzed by a machine learning model that enabled us to shed light on the basis of decision-making and physical mechanisms of the force tracking process. The finding from this work can control hydrodynamic force on the operation of fluidic pinball system and potentially pave the way for exploring efficient active flow control strategies in other complex fluid dynamic problems. △ Less

Submitted 22 April, 2023; originally announced April 2023.

SignReLU neural network and its approximation ability

Authors: Jianfei Li, Han Feng, Ding-Xuan Zhou

Abstract: Deep neural networks (DNNs) have garnered significant attention in various fields of science and technology in recent years. Activation functions define how neurons in DNNs process incoming signals for them. They are essential for learning non-linear transformations and for performing diverse computations among successive neuron layers. In the last few years, researchers have investigated the appr… ▽ More Deep neural networks (DNNs) have garnered significant attention in various fields of science and technology in recent years. Activation functions define how neurons in DNNs process incoming signals for them. They are essential for learning non-linear transformations and for performing diverse computations among successive neuron layers. In the last few years, researchers have investigated the approximation ability of DNNs to explain their power and success. In this paper, we explore the approximation ability of DNNs using a different activation function, called SignReLU. Our theoretical results demonstrate that SignReLU networks outperform rational and ReLU networks in terms of approximation performance. Numerical experiments are conducted comparing SignReLU with the existing activations such as ReLU, Leaky ReLU, and ELU, which illustrate the competitive practical performance of SignReLU. △ Less

Submitted 30 August, 2023; v1 submitted 18 October, 2022; originally announced October 2022.

Contrastive Psudo-supervised Classification for Intra-Pulse Modulation of Radar Emitter Signals Using data augmentation

Authors: HanCong Feng, XinHai Yan, KaiLi Jiang, XinYu Zhao, Bin Tang

Abstract: The automatic classification of radar waveform is a fundamental technique in electronic countermeasures (ECM).Recent supervised deep learning-based methods have achieved great success in a such classification task.However, those methods require enough labeled samples to work properly and in many circumstances, it is not available.To tackle this problem, in this paper, we propose a three-stages dee… ▽ More The automatic classification of radar waveform is a fundamental technique in electronic countermeasures (ECM).Recent supervised deep learning-based methods have achieved great success in a such classification task.However, those methods require enough labeled samples to work properly and in many circumstances, it is not available.To tackle this problem, in this paper, we propose a three-stages deep radar waveform clustering(DRSC) technique to automatically group the received signal samples without labels.Firstly, a pretext model is trained in a self-supervised way with the help of several data augmentation techniques to extract the class-dependent features.Next,the pseudo-supervised contrastive training is involved to further promote the separation between the extracted class-dependent features.And finally, the unsupervised problem is converted to a semi-supervised classification problem via pseudo label generation. The simulation results show that the proposed algorithm can effectively extract class-dependent features, outperforming several unsupervised clustering methods, even reaching performance on par with the supervised deep learning-based methods. △ Less

Submitted 13 October, 2022; originally announced October 2022.

Utilizing Explainable AI for improving the Performance of Neural Networks

Authors: Huawei Sun, Lorenzo Servadei, Hao Feng, Michael Stephan, Robert Wille, Avik Santra

Abstract: Nowadays, deep neural networks are widely used in a variety of fields that have a direct impact on society. Although those models typically show outstanding performance, they have been used for a long time as black boxes. To address this, Explainable Artificial Intelligence (XAI) has been develo** as a field that aims to improve the transparency of the model and increase their trustworthiness. W… ▽ More Nowadays, deep neural networks are widely used in a variety of fields that have a direct impact on society. Although those models typically show outstanding performance, they have been used for a long time as black boxes. To address this, Explainable Artificial Intelligence (XAI) has been develo** as a field that aims to improve the transparency of the model and increase their trustworthiness. We propose a retraining pipeline that consistently improves the model predictions starting from XAI and utilizing state-of-the-art techniques. To do that, we use the XAI results, namely SHapley Additive exPlanations (SHAP) values, to give specific training weights to the data samples. This leads to an improved training of the model and, consequently, better performance. In order to benchmark our method, we evaluate it on both real-life and public datasets. First, we perform the method on a radar-based people counting scenario. Afterward, we test it on the CIFAR-10, a public Computer Vision dataset. Experiments using the SHAP-based retraining approach achieve a 4% more accuracy w.r.t. the standard equal weight retraining for people counting tasks. Moreover, on the CIFAR-10, our SHAP-based weighting strategy ends up with a 3% accuracy rate than the training procedure with equal weighted samples. △ Less

Submitted 7 October, 2022; originally announced October 2022.

Comments: accepted at ICMLA 2022

Sampling of Correlated Bandlimited Continuous Signals by Joint Time-vertex Graph Fourier Transform

Authors: Zhongyi Ni, Feng Ji, Hang Sheng, Hui Feng, Bo Hu

Abstract: When sampling multiple signals, the correlation between the signals can be exploited to reduce the overall number of samples. In this paper, we study the sampling theory of multiple correlated signals, using correlation to sample them at the lowest sampling rate. Based on the correlation between signal sources, we model multiple continuous-time signals as continuous time-vertex graph signals. The… ▽ More When sampling multiple signals, the correlation between the signals can be exploited to reduce the overall number of samples. In this paper, we study the sampling theory of multiple correlated signals, using correlation to sample them at the lowest sampling rate. Based on the correlation between signal sources, we model multiple continuous-time signals as continuous time-vertex graph signals. The graph signals are projected onto orthogonal bases to remove spatial correlation and reduce dimensions by graph Fourier transform. When the bandwidths of the original signals and the reduced dimension signals are given, we prove the minimum sampling rate required for recovery of the original signals, and propose a feasible sampling scheme. △ Less

Submitted 10 October, 2022; originally announced October 2022.

Towards Efficient Modularity in Industrial Drying: A Combinatorial Optimization Viewpoint

Authors: Alisina Bayati, Amber Srivastava, Amir Malvandi, Hao Feng, Srinivasa Salapaka

Abstract: The industrial drying process consumes approximately 12% of the total energy used in manufacturing, with the potential for a 40% reduction in energy usage through improved process controls and the development of new drying technologies. To achieve cost-efficient and high-performing drying, multiple drying technologies can be combined in a modular fashion with optimal sequencing and control paramet… ▽ More The industrial drying process consumes approximately 12% of the total energy used in manufacturing, with the potential for a 40% reduction in energy usage through improved process controls and the development of new drying technologies. To achieve cost-efficient and high-performing drying, multiple drying technologies can be combined in a modular fashion with optimal sequencing and control parameters for each. This paper presents a mathematical formulation of this optimization problem and proposes a framework based on the Maximum Entropy Principle (MEP) to simultaneously solve for both optimal values of control parameters and optimal sequence. The proposed algorithm addresses the combinatorial optimization problem with a non-convex cost function riddled with multiple poor local minima. Simulation results on drying distillers dried grain (DDG) products show up to 12% improvement in energy consumption compared to the most efficient single-stage drying process. The proposed algorithm converges to local minima and is designed heuristically to reach the global minimum. △ Less

Submitted 5 April, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Learning of Dynamical Systems under Adversarial Attacks -- Null Space Property Perspective

Authors: Han Feng, Baturalp Yalcin, Javad Lavaei

Abstract: We study the identification of a linear time-invariant dynamical system affected by large-and-sparse disturbances modeling adversarial attacks or faults. Under the assumption that the states are measurable, we develop necessary and sufficient conditions for the recovery of the system matrices by solving a constrained lasso-type optimization problem. In addition, we provide an upper bound on the es… ▽ More We study the identification of a linear time-invariant dynamical system affected by large-and-sparse disturbances modeling adversarial attacks or faults. Under the assumption that the states are measurable, we develop necessary and sufficient conditions for the recovery of the system matrices by solving a constrained lasso-type optimization problem. In addition, we provide an upper bound on the estimation error whenever the disturbance sequence is a combination of small noise values and large adversarial values. Our results depend on the null space property that has been widely used in the lasso literature, and we investigate under what conditions this property holds for linear time-invariant dynamical systems. Lastly, we further study the conditions for a specific probabilistic model and support the results with numerical experiments. △ Less

Submitted 5 October, 2022; v1 submitted 4 October, 2022; originally announced October 2022.

Comments: 8 pages, 2 figures

MSC Class: 93

Spherical Image Inpainting with Frame Transformation and Data-driven Prior Deep Networks

Authors: Jianfei Li, Chaoyan Huang, Raymond Chan, Han Feng, Micheal Ng, Tieyong Zeng

Abstract: Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modelling, and medical imaging. It is non-trivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with deep learning-based regularizer. Instead of a naive… ▽ More Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modelling, and medical imaging. It is non-trivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regularizer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using deep learning denoiser and plug-and-play model. △ Less

Submitted 29 September, 2022; originally announced September 2022.

MSC Class: 68Q25; 68R10; 68U05

Learnability Enhancement for Low-light Raw Denoising: Where Paired Real Data Meets Noise Modeling

Authors: Hansen Feng, Lizhi Wang, Yuzhi Wang, Hua Huang

Abstract: Low-light raw denoising is an important and valuable task in computational photography where learning-based methods trained with paired real data are mainstream. However, the limited data volume and complicated noise distribution have constituted a learnability bottleneck for paired real data, which limits the denoising performance of learning-based methods. To address this issue, we present a lea… ▽ More Low-light raw denoising is an important and valuable task in computational photography where learning-based methods trained with paired real data are mainstream. However, the limited data volume and complicated noise distribution have constituted a learnability bottleneck for paired real data, which limits the denoising performance of learning-based methods. To address this issue, we present a learnability enhancement strategy to reform paired real data according to noise modeling. Our strategy consists of two efficient techniques: shot noise augmentation (SNA) and dark shading correction (DSC). Through noise model decoupling, SNA improves the precision of data map** by increasing the data volume and DSC reduces the complexity of data map** by reducing the noise complexity. Extensive results on the public datasets and real imaging scenarios collectively demonstrate the state-of-the-art performance of our method. Our code is available at: https://github.com/megvii-research/PMN. △ Less

Submitted 18 August, 2022; v1 submitted 13 July, 2022; originally announced July 2022.

Comments: 9 pages, 8 figures, to be published in ACMMM 2022

Cross-modal Learning of Graph Representations using Radar Point Cloud for Long-Range Gesture Recognition

Authors: Souvik Hazra, Hao Feng, Gamze Naz Kiprit, Michael Stephan, Lorenzo Servadei, Robert Wille, Robert Weigel, Avik Santra

Abstract: Gesture recognition is one of the most intuitive ways of interaction and has gathered particular attention for human computer interaction. Radar sensors possess multiple intrinsic properties, such as their ability to work in low illumination, harsh weather conditions, and being low-cost and compact, making them highly preferable for a gesture recognition solution. However, most literature work foc… ▽ More Gesture recognition is one of the most intuitive ways of interaction and has gathered particular attention for human computer interaction. Radar sensors possess multiple intrinsic properties, such as their ability to work in low illumination, harsh weather conditions, and being low-cost and compact, making them highly preferable for a gesture recognition solution. However, most literature work focuses on solutions with a limited range that is lower than a meter. We propose a novel architecture for a long-range (1m - 2m) gesture recognition solution that leverages a point cloud-based cross-learning approach from camera point cloud to 60-GHz FMCW radar point cloud, which allows learning better representations while suppressing noise. We use a variant of Dynamic Graph CNN (DGCNN) for the cross-learning, enabling us to model relationships between the points at a local and global level and to model the temporal dynamics a Bi-LSTM network is employed. In the experimental results section, we demonstrate our model's overall accuracy of 98.4% for five gestures and its generalization capability. △ Less

Submitted 19 May, 2022; v1 submitted 31 March, 2022; originally announced March 2022.

Comments: Accepted by IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM 2022)

Convolutional Neural Networks for Spherical Signal Processing via Spherical Haar Tight Framelets

Authors: Jianfei Li, Han Feng, Xiaosheng Zhuang

Abstract: In this paper, we develop a general theoretical framework for constructing Haar-type tight framelets on any compact set with a hierarchical partition. In particular, we construct a novel area-regular hierarchical partition on the 2-sphere and establish its corresponding spherical Haar tight framelets with directionality. We conclude by evaluating and illustrating the effectiveness of our area-regu… ▽ More In this paper, we develop a general theoretical framework for constructing Haar-type tight framelets on any compact set with a hierarchical partition. In particular, we construct a novel area-regular hierarchical partition on the 2-sphere and establish its corresponding spherical Haar tight framelets with directionality. We conclude by evaluating and illustrating the effectiveness of our area-regular spherical Haar tight framelets in several denoising experiments. Furthermore, we propose a convolutional neural network (CNN) model for spherical signal denoising which employs the fast framelet decomposition and reconstruction algorithms. Experiment results show that our proposed CNN model outperforms threshold methods, and processes strong generalization and robustness properties. △ Less

Submitted 17 January, 2022; originally announced January 2022.

Sensoring and Application of Multimodal Data for the Detection of Freezing of Gait in Parkinson's Disease

Authors: Wei Zhang, Debin Huang, Hantao Li, Lipeng Wang, Yanzhao Wei, Kang Pan, Lin Ma, Huanhuan Feng, **g Pan, Yuzhu Guo

Abstract: The accurate and reliable detection or prediction of freezing of gaits (FOG) is important for fall prevention in Parkinson's Disease (PD) and studying the physiological transitions during the occurrence of FOG. Integrating both commercial and self-designed sensors, a protocal has been designed to acquire multimodal physical and physiological information during FOG, including gait acceleration (ACC… ▽ More The accurate and reliable detection or prediction of freezing of gaits (FOG) is important for fall prevention in Parkinson's Disease (PD) and studying the physiological transitions during the occurrence of FOG. Integrating both commercial and self-designed sensors, a protocal has been designed to acquire multimodal physical and physiological information during FOG, including gait acceleration (ACC), electroencephalogram (EEG), electromyogram (EMG), and skin conductance (SC). Two tasks were designed to trigger FOG, including gait initiation failure and FOG during walking. A total number of 12 PD patients completed the experiments and produced a total length of 3 hours and 42 minutes of valid data. The FOG episodes were labeled by two qualified physicians. Each unimodal data and combinations have been used to detect FOG. Results showed that multimodal data benefit the detection of FOG. Among unimodal data, EEG had better discriminative ability than ACC and EMG. However, the acquisition of EEG are more complicated. Multimodal motional and electrophysiological data can also be used to study the physiological transition process during the occurrence of FOG and provide personalised interventions. △ Less

Submitted 8 October, 2021; originally announced October 2021.

Comments: This paper has 13 pages and 8 figures. The data was published on Mendeley Data, where raw data availible at https://data.mendeley.com/datasets/t8j8v4hnm4/1 and filtered data availible at https://data.mendeley.com/datasets/r8gmbtv7w2/3

Recovery of Graph Signals from Sign Measurements

Authors: Wenwei Liu, Hui Feng, Kaixuan Wang, Feng Ji, Bo Hu

Abstract: Sampling and interpolation have been extensively studied, in order to reconstruct or estimate the entire graph signal from the signal values on a subset of vertexes, of which most achievements are about continuous signals. While in a lot of signal processing tasks, signals are not fully observed, and only the signs of signals are available, for example a rating system may only provide several simp… ▽ More Sampling and interpolation have been extensively studied, in order to reconstruct or estimate the entire graph signal from the signal values on a subset of vertexes, of which most achievements are about continuous signals. While in a lot of signal processing tasks, signals are not fully observed, and only the signs of signals are available, for example a rating system may only provide several simple options. In this paper, the reconstruction of band-limited graph signals based on sign sampling is discussed and a greedy sampling strategy is proposed. The simulation experiments are presented, and the greedy sampling algorithm is compared with random sampling algorithm, which verify the validity of the proposed approach. △ Less

Submitted 26 September, 2021; originally announced September 2021.

The Incubator Case Study for Digital Twin Engineering

Authors: Hao Feng, Cláudio Gomes, Casper Thule, Kenneth Lausdahl, Michael Sandberg, Peter Gorm Larsen

Abstract: To demystify the Digital Twin concept, we built a simple yet representative thermal incubator system. The incubator is an insulated box fitted with a heatbed, and complete with a software system for communication, a controller, and simulation models. We developed two simulation models to predict the temperature inside the incubator, one with two free parameters and one with four free parameters. O… ▽ More To demystify the Digital Twin concept, we built a simple yet representative thermal incubator system. The incubator is an insulated box fitted with a heatbed, and complete with a software system for communication, a controller, and simulation models. We developed two simulation models to predict the temperature inside the incubator, one with two free parameters and one with four free parameters. Our experiments showed that the latter model was better at predicting the thermal inertia of the heatbed itself, which makes it more appropriate for further development of the digital twin. The hardware and software used in this case study are available open source, providing an accessible platform for those who want to develop and verify their own techniques for digital twins. △ Less

Submitted 20 February, 2021; originally announced February 2021.

Comments: 18 pages, 13 figures

Regularized Recovery by Multi-order Partial Hypergraph Total Variation

Authors: Ruyuan Qu, Jiaqi He, Hui Feng, Chongbin Xu, Bo Hu

Abstract: Capturing complex high-order interactions among data is an important task in many scenarios. A common way to model high-order interactions is to use hypergraphs whose topology can be mathematically represented by tensors. Existing methods use a fixed-order tensor to describe the topology of the whole hypergraph, which ignores the divergence of different-order interactions. In this work, we take th… ▽ More Capturing complex high-order interactions among data is an important task in many scenarios. A common way to model high-order interactions is to use hypergraphs whose topology can be mathematically represented by tensors. Existing methods use a fixed-order tensor to describe the topology of the whole hypergraph, which ignores the divergence of different-order interactions. In this work, we take this divergence into consideration, and propose a multi-order hypergraph Laplacian and the corresponding total variation. Taking this total variation as a regularization term, we can utilize the topology information contained by it to smooth the hypergraph signal. This can help distinguish different-order interactions and represent high-order interactions accurately. △ Less

Submitted 19 February, 2021; originally announced February 2021.

Comments: ICASSP 2021

Boundary Stabilization and Observation of an Unstable Heat Equation in a General Multi-dimensional Domain

Authors: Hongyin** Feng, Pei-Hua Lang, Jiankang Liu

Abstract: In this paper, we consider the exponential stabilization and observation of an unstable heat equation in a general multi-dimensional domain by combining the finite-dimensional spectral truncation technique and the recently developed dynamics compensation approach. In contrast to the unstable one-dimensional partial differential equation (PDE), such as the transport equation, wave equation and the… ▽ More In this paper, we consider the exponential stabilization and observation of an unstable heat equation in a general multi-dimensional domain by combining the finite-dimensional spectral truncation technique and the recently developed dynamics compensation approach. In contrast to the unstable one-dimensional partial differential equation (PDE), such as the transport equation, wave equation and the heat equation, that can be treated by the well-known PDE backstep** method, stabilization of unstable PDE in a general multi-dimensional domain is still a challenging problem. We treat the stabilization and observation problems separately. A dynamical state feedback law is proposed firstly to stabilize the unstable heat equation exponentially and then a state observer is designed via a boundary measurement. Both the stability of the closed-loop system and the well-posedness of the observer are proved. Some of the theoretical results are validated by the numerical simulations. △ Less

Submitted 4 February, 2021; originally announced February 2021.

Comments: 23 pages,8 figures

MSC Class: 93C05; 93B52; 47A62

SharpGAN: Receptive Field Block Net for Dynamic Scene Deblurring

Authors: Hui Feng, Jundong Guo, Sam Shuzhi Ge

Abstract: When sailing at sea, the smart ship will inevitably produce swaying motion due to the action of wind, wave and current, which makes the image collected by the visual sensor appear motion blur. This will have an adverse effect on the object detection algorithm based on the vision sensor, thereby affect the navigation safety of the smart ship. In order to remove the motion blur in the images during… ▽ More When sailing at sea, the smart ship will inevitably produce swaying motion due to the action of wind, wave and current, which makes the image collected by the visual sensor appear motion blur. This will have an adverse effect on the object detection algorithm based on the vision sensor, thereby affect the navigation safety of the smart ship. In order to remove the motion blur in the images during the navigation of the smart ship, we propose SharpGAN, a new image deblurring method based on the generative adversarial network. First of all, the Receptive Field Block Net (RFBNet) is introduced to the deblurring network to strengthen the network's ability to extract the features of blurred image. Secondly, we propose a feature loss that combines different levels of image features to guide the network to perform higher-quality deblurring and improve the feature similarity between the restored images and the sharp image. Finally, we propose to use the lightweight RFB-s module to improve the real-time performance of deblurring network. Compared with the existing deblurring methods on large-scale real sea image datasets and large-scale deblurring datasets, the proposed method not only has better deblurring performance in visual perception and quantitative criteria, but also has higher deblurring efficiency. △ Less

Submitted 30 December, 2020; originally announced December 2020.

Comments: 15 pages, 6 figures

ACM Class: I.2.10

Extended Dynamics Observer for Linear Systems with Disturbance

Authors: Hongyin** Feng, Bao-Zhu Guo

Abstract: This is the last part of four series papers, aiming at stabilization for signal-input-signaloutput (SISO) linear finite-dimensional systems corrupted by general input disturbances. A new observer, referred to as Extended Dynamics Observer (EDO), is proposed to estimate both the state and disturbance simultaneously. The working mechanism of EDO consists of two parts: The disturbance with known dyna… ▽ More This is the last part of four series papers, aiming at stabilization for signal-input-signaloutput (SISO) linear finite-dimensional systems corrupted by general input disturbances. A new observer, referred to as Extended Dynamics Observer (EDO), is proposed to estimate both the state and disturbance simultaneously. The working mechanism of EDO consists of two parts: The disturbance with known dynamics is canceled completely by its dynamics and the disturbance with unknown dynamics is absorbed by high-gain. It is found that the high-gain is always working as long as the control plant with unknown input disturbance is observable which is the only assumption for the observer design. When the disturbance dynamics are completely unknown except some boundedness, the EDO is reduced to an extension of the well-known extended state observer or high-gain observer. The main advantage of the developed method is that the prior information about both the control plant and the disturbance can be utilized as much as possible. The more the prior information we have, the better performance the observer would be. An EDO based stabilizing output feedback is also developed in the spirit of estimation/cancellation strategy. The stability of the resulting closed-loop system is established and some of the theoretical results are validated by numerical simulations. △ Less

Submitted 12 November, 2020; originally announced November 2020.

Comments: 31 pages, 4 figures, this is the last part of the series works: dynamics compensation for linear systems

MSC Class: 93C05; 93B52; 47A62

Sampling Theory of Bandlimited Continuous-Time Graph Signals

Authors: Feng Ji, Hui Feng, Hang Sheng, Wee Peng Tay

Abstract: A continuous-time graph signal can be viewed as a time series of graph signals. It generalizes both the classical continuous-time signal and ordinary graph signal. Therefore, such a signal can be considered as a function on two domains: the graph domain and the time domain. In this paper, we consider the sampling theory of bandlimited continuous-time graph signals. To formulate the sampling proble… ▽ More A continuous-time graph signal can be viewed as a time series of graph signals. It generalizes both the classical continuous-time signal and ordinary graph signal. Therefore, such a signal can be considered as a function on two domains: the graph domain and the time domain. In this paper, we consider the sampling theory of bandlimited continuous-time graph signals. To formulate the sampling problem, we need to consider the interaction between the graph and time domains. We describe an explicit procedure to determine a discrete sampling set for perfect signal recovery. Moreover, in analogous to the Nyquist-Shannon sampling theorem, we give an explicit formula for the minimal sample rate. △ Less

Submitted 1 October, 2021; v1 submitted 19 October, 2020; originally announced October 2020.

Delay Compensation for Regular Linear Systems

Authors: Hongyin** Feng

Abstract: This is the third part of four series papers, aiming at the delay compensation for the abstract linear system (A,B,C). Both the input delay and output delay are investigated. We first propose a full state feedback control to stabilize the system (A,B) with input delay and then design a Luenberger-like observer for the system (A,C) in terms of the delayed output. We formulate the delay compensation… ▽ More This is the third part of four series papers, aiming at the delay compensation for the abstract linear system (A,B,C). Both the input delay and output delay are investigated. We first propose a full state feedback control to stabilize the system (A,B) with input delay and then design a Luenberger-like observer for the system (A,C) in terms of the delayed output. We formulate the delay compensation in the framework of regular linear systems. The developed approach builds upon an upper-block-triangle transform that is associated with a Sylvester operator equation. It is found that the controllability/observability map of system (-A,B)/(-A,-C) happens to be the solution of the corresponding Sylvester equation. As an immediate consequence, both the feedback law and the state observer can be expressed explicitly in the operator form. The exponential stability of the resulting closed-loop system and the exponential convergence of the observation error are established without using the Lyapunov functional approach. The theoretical results are validated through the delay compensation for a benchmark one-dimensional wave equation. △ Less

Submitted 4 September, 2020; originally announced September 2020.

Comments: 28 pages, this is the third part of the series works: dynamics compensation for linear systems

MSC Class: 93C05; 93B52; 47A62

Dynamics Compensation in Observation of Abstract Linear Systems

Authors: Hongyin** Feng, Xiao-Hui Wu, Bao-Zhu Guo

Abstract: This is the second part of four series papers, aiming at the problem of sensor dynamics compensation for abstract linear systems. Two major issues are addressed. The first one is about the sensor dynamics compensation in system observation and the second one is on the disturbance dynamics compensation in output regulation for linear system. Both of them can be described by the problem of state obs… ▽ More This is the second part of four series papers, aiming at the problem of sensor dynamics compensation for abstract linear systems. Two major issues are addressed. The first one is about the sensor dynamics compensation in system observation and the second one is on the disturbance dynamics compensation in output regulation for linear system. Both of them can be described by the problem of state observation for an abstract cascade system. We consider these two apparently different problems from the same abstract linear system point of view. A new scheme of the observer design for the abstract cascade system is developed and the exponential convergence of the observation error is established. It is shown that the error based observer design in the problem of output regulation can be converted into a sensor dynamics compensation problem by the well known regulator equations. As a result, a tracking error based observer for output regulation problem is designed by exploiting the developed method. As applications, the ordinary differential equations (ODEs) with output time-delay and an unstable heat equation with ODE sensor dynamics are fully investigated to validate the theoretical results. The numerical simulations for the unstable heat system are carried out to validate the proposed method visually. △ Less

Submitted 3 September, 2020; originally announced September 2020.

Comments: 29 pages, 2 figures, this is the second part of the series works: dynamics compensation for linear systems

MSC Class: 93C05; 93B52; 47A62

Actuator Dynamics Compensation in Stabilization of Abstract Linear Systems

Authors: Hongyin** Feng, Xiao-Hui Wu, Bao-Zhu Guo

Abstract: This is the first part of four series papers, aiming at the problem of actuator dynamics compensation for linear systems. We consider the stabilization of a type of cascade abstract linear systems which model the actuator dynamics compensation for linear systems where both the control plant and its actuator dynamics can be infinite-dimensional. We develop a systematic way to stabilize the cascade… ▽ More This is the first part of four series papers, aiming at the problem of actuator dynamics compensation for linear systems. We consider the stabilization of a type of cascade abstract linear systems which model the actuator dynamics compensation for linear systems where both the control plant and its actuator dynamics can be infinite-dimensional. We develop a systematic way to stabilize the cascade systems by a full state feedback. Both the well-posedness and the exponential stability of the resulting closed-loop system are established in the abstract framework. A sufficient condition of the existence of compensator for ordinary differential equation (ODE) with partial differential equation (PDE) actuator dynamics is obtained. The feedback design is based on a novelly constructed upper-block-triangle transform and the Lyapunov function design is not needed in the stability analysis. As applications, an ODE with input delay and an unstable heat equation with ODE actuator dynamics are investigated to validate the theoretical results. The numerical simulations for the unstable heat system are carried out to validate the proposed approach visually. △ Less

Submitted 25 August, 2020; originally announced August 2020.

Comments: 27 pages, 5 figures, this is the first part of the series works: dynamics compensation for linear systems

MSC Class: 93C05; 93B52; 47A62

Modular Medium Voltage AC to Low Voltage DC Converter for Extreme Fast Charging Applications

Authors: M A Awal, Iqbal Husain, Md Rashed Hassan Bipu, Oscar Andreas Montes, Fei Teng, Hao Feng, Mehnaz Khan, Srdjan Lukic

Abstract: A modular and scalable converter for medium voltage (MV) AC to low voltage (LV) DC power conversion is proposed; single-phase-modules (SPMs), each consisting of an active-front-end (AFE) stage and an isolated DC-DC stage, are connected in input-series-output-parallel (ISOP) configuration to reach desired voltage and power capacity. In prior art, high-speed bidirectional communication among modules… ▽ More A modular and scalable converter for medium voltage (MV) AC to low voltage (LV) DC power conversion is proposed; single-phase-modules (SPMs), each consisting of an active-front-end (AFE) stage and an isolated DC-DC stage, are connected in input-series-output-parallel (ISOP) configuration to reach desired voltage and power capacity. In prior art, high-speed bidirectional communication among modules and a centralized controller is required to ensure module-level voltage and power balancing, which severely limits the scalability and practical realization of higher voltage and higher power systems. Moreover, large capacitors are used to suppress double-line-frequency voltage variations on the common MV DC bus shared by the AFE and the DC-DC stage originating from AC power pulsations through the SPMs. We propose a comprehensive controller which achieves voltage and power balancing using complete decentralized control of the DC-DC stages based on only local sensor feedback and the AFE stages are controlled using feedback of only the LV DC output. Furthermore, reduced capacitor requirement on the MV DC bus is achieved through design and control. The proposed method is validated through simulation and experimental results. △ Less

Submitted 8 July, 2020; originally announced July 2020.

Comments: 8 pages; 14 figures; Component level validations through hardware experiments have been included in this manuscript; full-system validation is ongoing and will be reported in the final version while submitting for peer review to IEEE Transactions on Power Electronics or IEEE Journal of Emerging and Selected Topics in Power Electronics

Low-light Image Restoration with Short- and Long-exposure Raw Pairs

Authors: Meng Chang, Huajun Feng, Zhihai Xu, Qi Li

Abstract: Low-light imaging with handheld mobile devices is a challenging issue. Limited by the existing models and training data, most existing methods cannot be effectively applied in real scenarios. In this paper, we propose a new low-light image restoration method by using the complementary information of short- and long-exposure images. We first propose a novel data generation method to synthesize real… ▽ More Low-light imaging with handheld mobile devices is a challenging issue. Limited by the existing models and training data, most existing methods cannot be effectively applied in real scenarios. In this paper, we propose a new low-light image restoration method by using the complementary information of short- and long-exposure images. We first propose a novel data generation method to synthesize realistic short- and longexposure raw images by simulating the imaging pipeline in lowlight environment. Then, we design a new long-short-exposure fusion network (LSFNet) to deal with the problems of low-light image fusion, including high noise, motion blur, color distortion and misalignment. The proposed LSFNet takes pairs of shortand long-exposure raw images as input, and outputs a clear RGB image. Using our data generation method and the proposed LSFNet, we can recover the details and color of the original scene, and improve the low-light image quality effectively. Experiments demonstrate that our method can outperform the state-of-the art methods. △ Less

Submitted 28 February, 2021; v1 submitted 30 June, 2020; originally announced July 2020.

Beyond Camera Motion Blur Removing: How to Handle Outliers in Deblurring

Authors: Meng Chang, Chenwei Yang, Huajun Feng, Zhihai Xu, Qi Li

Abstract: Camera motion deblurring is an important low-level vision task for achieving better imaging quality. When a scene has outliers such as saturated pixels, the captured blurred image becomes more difficult to restore. In this paper, we propose a novel method to handle camera motion blur with outliers. We first propose an edge-aware scale-recurrent network (EASRN) to conduct deblurring. EASRN has a se… ▽ More Camera motion deblurring is an important low-level vision task for achieving better imaging quality. When a scene has outliers such as saturated pixels, the captured blurred image becomes more difficult to restore. In this paper, we propose a novel method to handle camera motion blur with outliers. We first propose an edge-aware scale-recurrent network (EASRN) to conduct deblurring. EASRN has a separate deblurring module that removes blur at multiple scales and an upsampling module that fuses different input scales. Then a salient edge detection network is proposed to supervise the training process and constraint the edges restoration. By simulating camera motion and adding various light sources, we can generate blurred images with saturation cutoff. Using the proposed data generation method, our network can learn to deal with outliers effectively. We evaluate our method on public test datasets including the GoPro dataset, Kohler's dataset and Lai's dataset. Both objective evaluation indexes and subjective visualization show that our method results in better deblurring quality than other state-of-the-art approaches. △ Less

Submitted 27 April, 2021; v1 submitted 24 February, 2020; originally announced February 2020.

Sampling Policy Design for Tracking Time-Varying Graph Signals with Adaptive Budget Allocation

Authors: Xuan Xie, Hui Feng, Bo Hu

Abstract: There have been many works that focus on the sampling set design for a static graph signal, but few for time-varying graph signals (GS). In this paper, we concentrate on how to select vertices to sample and how to allocate the sampling budget for a time-varying GS to achieve a minimal tracking error for the long-term. In the Kalman Filter (KF) framework, the problem of sampling policy design and b… ▽ More There have been many works that focus on the sampling set design for a static graph signal, but few for time-varying graph signals (GS). In this paper, we concentrate on how to select vertices to sample and how to allocate the sampling budget for a time-varying GS to achieve a minimal tracking error for the long-term. In the Kalman Filter (KF) framework, the problem of sampling policy design and budget allocation is formulated as an infinite horizon sequential decision process, in which the optimal sampling policy is obtained by Dynamic Programming (DP). Since the optimal policy is intractable, an approximate algorithm is proposed by truncating the infinite horizon. By introducing a new tool for analyzing the convexity or concavity of composite functions, we prove that the truncated problem is convex. Finally, we demonstrate the performance of the proposed approach through numerical experiments. △ Less

Submitted 22 October, 2020; v1 submitted 14 February, 2020; originally announced February 2020.

Comments: 6 pages, 4 figures, accepted by APSIPA 2020

Spatial-Adaptive Network for Single Image Denoising

Authors: Meng Chang, Qi Li, Huajun Feng, Zhihai Xu

Abstract: Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but more computational overhead is needed. In this paper, we propose a novel spatial-adaptive denoising network (SADN… ▽ More Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but more computational overhead is needed. In this paper, we propose a novel spatial-adaptive denoising network (SADNet) for efficient single image blind noise removal. To adapt to changes in spatial textures and edges, we design a residual spatial-adaptive block. Deformable convolution is introduced to sample the spatially correlated features for weighting. An encoder-decoder structure with a context block is introduced to capture multiscale information. With noise removal from the coarse to fine, a high-quality noisefree image can be obtained. We apply our method to both synthetic and real noisy image datasets. The experimental results demonstrate that our method can surpass the state-of-the-art denoising methods both quantitatively and visually. △ Less

Submitted 13 July, 2020; v1 submitted 28 January, 2020; originally announced January 2020.

Fast Color-guided Depth Denoising for RGB-D Images by Graph Filtering

Authors: Qiwei Huang, Ruikang Li, Zidong Jiang, Wei Feng, Sijie Lin, Hui Feng, Bo Hu

Abstract: Depth images captured by off-the-shelf RGB-D cameras suffer from much stronger noise than color images. In this paper, we propose a method to denoise the depth images in RGB-D images by color-guided graph filtering. Our iterative method contains two components: color-guided similarity graph construction, and graph filtering on the depth signal. Implemented in graph vertex domain, filtering is acce… ▽ More Depth images captured by off-the-shelf RGB-D cameras suffer from much stronger noise than color images. In this paper, we propose a method to denoise the depth images in RGB-D images by color-guided graph filtering. Our iterative method contains two components: color-guided similarity graph construction, and graph filtering on the depth signal. Implemented in graph vertex domain, filtering is accelerated as computation only occurs among neighboring vertices. Experimental results show that our method outperforms state-of-art depth image denoising methods significantly both on quality and efficiency. △ Less

Submitted 7 December, 2019; v1 submitted 4 December, 2019; originally announced December 2019.

Comments: 5 pages, 4 figures

Bayesian Design of Sampling Set for Bandlimited Graph Signals

Authors: Xuan Xie, Junhao Yu, Hui Feng, Bo Hu

Abstract: The design of sampling set (DoS) for bandlimited graph signals (GS) has been extensively studied in recent years, but few of them exploit the benefits of the stochastic prior of GS. In this work, we introduce the optimization framework for Bayesian DoS of bandlimited GS. We also illustrate how the choice of different sampling sets affects the estimation error and how the prior knowledge influences… ▽ More The design of sampling set (DoS) for bandlimited graph signals (GS) has been extensively studied in recent years, but few of them exploit the benefits of the stochastic prior of GS. In this work, we introduce the optimization framework for Bayesian DoS of bandlimited GS. We also illustrate how the choice of different sampling sets affects the estimation error and how the prior knowledge influences the result of DoS compared with the non-Bayesian DoS by the aid of analyzing Gershgorin discs of error metric matrix. Finally, based on our analysis, we propose a heuristic algorithm for DoS to avoid solving the optimization problem directly. △ Less

Submitted 7 September, 2019; originally announced September 2019.

Comments: Accepted by GloalSIP2019

On Critical Sampling of Time-Vertex Graph Signals

Authors: Junhao Yu, Xuan Xie, Hui Feng, Bo Hu

Abstract: Joint time-vertex graph signals are pervasive in real-world. This paper focuses on the fundamental problem of sampling and reconstruction of joint time-vertex graph signals. We prove the existence and the necessary condition of a critical sampling set using minimum number of samples in time and graph domain respectively. The theory proposed in this paper suggests to assign heterogeneous sampling p… ▽ More Joint time-vertex graph signals are pervasive in real-world. This paper focuses on the fundamental problem of sampling and reconstruction of joint time-vertex graph signals. We prove the existence and the necessary condition of a critical sampling set using minimum number of samples in time and graph domain respectively. The theory proposed in this paper suggests to assign heterogeneous sampling pattern for each node in a network under the constraint of minimum resources. An efficient algorithm is also provided to construct a critical sampling set. △ Less

Submitted 19 November, 2019; v1 submitted 5 September, 2019; originally announced September 2019.

Comments: GlobalSIP 2019

An Interactive Insight Identification and Annotation Framework for Power Grid Pixel Maps using DenseU-Hierarchical VAE

Authors: Tianye Zhang, Haozhe Feng, Zexian Chen, Can Wang, Yanhao Huang, Yong Tang, Wei Chen

Abstract: Insights in power grid pixel maps (PGPMs) refer to important facility operating states and unexpected changes in the power grid. Identifying insights helps analysts understand the collaboration of various parts of the grid so that preventive and correct operations can be taken to avoid potential accidents. Existing solutions for identifying insights in PGPMs are performed manually, which may be la… ▽ More Insights in power grid pixel maps (PGPMs) refer to important facility operating states and unexpected changes in the power grid. Identifying insights helps analysts understand the collaboration of various parts of the grid so that preventive and correct operations can be taken to avoid potential accidents. Existing solutions for identifying insights in PGPMs are performed manually, which may be laborious and expertise-dependent. In this paper, we propose an interactive insight identification and annotation framework by leveraging an enhanced variational autoencoder (VAE). In particular, a new architecture, DenseU-Hierarchical VAE (DUHiV), is designed to learn representations from large-sized PGPMs, which achieves a significantly tighter evidence lower bound (ELBO) than existing Hierarchical VAEs with a Multilayer Perceptron architecture. Our approach supports modulating the derived representations in an interactive visual interface, discover potential insights and create multi-label annotations. Evaluations using real-world PGPMs datasets show that our framework outperforms the baseline models in identifying and annotating insights. △ Less

Submitted 22 May, 2019; originally announced May 2019.

Active Sampling for Approximately Bandlimited Graph Signals

Authors: Sijie Lin, Xuan Xie, Hui Feng, Bo Hu

Abstract: This paper investigates the active sampling for estimation of approximately bandlimited graph signals. With the assistance of a graph filter, an approximately bandlimited graph signal can be formulated by a Gaussian random field over the graph. In contrast to offline sampling set design methods which usually rely on accurate prior knowledge about the model, unknown parameters in signal and noise d… ▽ More This paper investigates the active sampling for estimation of approximately bandlimited graph signals. With the assistance of a graph filter, an approximately bandlimited graph signal can be formulated by a Gaussian random field over the graph. In contrast to offline sampling set design methods which usually rely on accurate prior knowledge about the model, unknown parameters in signal and noise distribution are allowed in the proposed active sampling algorithm. The active sampling process is divided into two alternating stages: unknown parameters are first estimated by Expectation Maximization (EM), with which the next node to sample is selected based on historical observations according to predictive uncertainty. Validated by simulations compared with related approaches, the proposed algorithm can reduce the sample size to reach a certain estimation accuracy. △ Less

Submitted 16 February, 2019; v1 submitted 12 February, 2019; originally announced February 2019.

Comments: ICASSP 2019

Robust Beamforming for Downlink 3D-MIMO Systems with $l_1$-norm Bounded CSI Uncertainty

Authors: Kai Liu, Hui Feng, Tao Yang, Bo Hu

Abstract: In this paper, a novel robust beamforming scheme is proposed in three dimensional multi-input multi-output (3D-MIMO) systems. As one of the typical deployments of massive MIMO, a 3D-MIMO system owns sparse channels in angular domain. Thus, various of sparse channel estimation algorithms produce sparse channel estimation errors which can be utilized to narrow down the perturbation region of imperfe… ▽ More In this paper, a novel robust beamforming scheme is proposed in three dimensional multi-input multi-output (3D-MIMO) systems. As one of the typical deployments of massive MIMO, a 3D-MIMO system owns sparse channels in angular domain. Thus, various of sparse channel estimation algorithms produce sparse channel estimation errors which can be utilized to narrow down the perturbation region of imperfect CSI. We investigate a $l_1$-norm bounded channel uncertainty model for the robust beamforming problems, which captures the sparse nature of channel errors. Compared with the conventional spherical uncertainty, we prove that the scheme with $l_1$-norm bounded uncertainty consumes less beamforming power with the same signal to interference and noise ratio (SINR) thresholds. The proposed scheme is reformulated as a second-order cone programming (SOCP) and simulation results verify the effectiveness of our algorithm. △ Less

Submitted 1 November, 2018; originally announced December 2018.

Comments: Accepted by WCSP 2018, Hangzhou, China