Showing 1–25 of 25 results for author: Yi, Z

Real-World Computational Aberration Correction via Quantized Domain-Mixing Representation

Authors: Qi Jiang, Zhonghua Yi, Shaohua Gao, Yao Gao, Xiaolong Qian, Hao Shi, Lei Sun, Zhijie Xu, Kailun Yang, Kaiwei Wang

Abstract: Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications. In this paper, in contrast to improving the simulation pipeline, we deliver a novel insight into real-world CAC from the perspective of Unsupervi… ▽ More Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications. In this paper, in contrast to improving the simulation pipeline, we deliver a novel insight into real-world CAC from the perspective of Unsupervised Domain Adaptation (UDA). By incorporating readily accessible unpaired real-world data into training, we formalize the Domain Adaptive CAC (DACAC) task, and then introduce a comprehensive Real-world aberrated images (Realab) dataset to benchmark it. The setup task presents a formidable challenge due to the intricacy of understanding the target aberration domain. To this intent, we propose a novel Quntized Domain-Mixing Representation (QDMR) framework as a potent solution to the issue. QDMR adapts the CAC model to the target domain from three key aspects: (1) reconstructing aberrated images of both domains by a VQGAN to learn a Domain-Mixing Codebook (DMC) which characterizes the degradation-aware priors; (2) modulating the deep features in CAC model with DMC to transfer the target domain knowledge; and (3) leveraging the trained VQGAN to generate pseudo target aberrated images from the source ones for convincing target domain supervision. Extensive experiments on both synthetic and real-world benchmarks reveal that the models with QDMR consistently surpass the competitive methods in mitigating the synthetic-to-real gap, which produces visually pleasant real-world CAC results with fewer artifacts. Codes and datasets will be made publicly available. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Comments: Codes and datasets will be made publicly available at https://github.com/zju-jiangqi/QDMR

3D Spectrum Map** and Reconstruction under Multi-Radiation Source Scenarios

Authors: Wang Jie, Lin Zhipeng, Zhu Qiuming, Wu Qihui, Lan Tianxu, Zhao Yi, Bai Yunpeng, Zhong Weizhi

Abstract: Spectrum map construction, which is crucial in cognitive radio (CR) system, visualizes the invisible space of the electromagnetic spectrum for spectrum-resource management and allocation. Traditional reconstruction methods are generally for two-dimensional (2D) spectrum map and driven by abundant sampling data. In this paper, we propose a data-model-knowledge-driven reconstruction scheme to constr… ▽ More Spectrum map construction, which is crucial in cognitive radio (CR) system, visualizes the invisible space of the electromagnetic spectrum for spectrum-resource management and allocation. Traditional reconstruction methods are generally for two-dimensional (2D) spectrum map and driven by abundant sampling data. In this paper, we propose a data-model-knowledge-driven reconstruction scheme to construct the three-dimensional (3D) spectrum map under multi-radiation source scenarios. We firstly design a maximum and minimum path loss difference (MMPLD) clustering algorithm to detect the number of radiation sources in a 3D space. Then, we develop a joint location-power estimation method based on the heuristic population evolutionary optimization algorithm. Considering the variation of electromagnetic environment, we self-learn the path loss (PL) model based on the sampling data. Finally, the 3D spectrum is reconstructed according to the self-learned PL model and the extracted knowledge of radiation sources. Simulations show that the proposed 3D spectrum map reconstruction scheme not only has splendid adaptability to the environment, but also achieves high spectrum construction accuracy even when the sampling rate is very low. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Multimodal Indoor Localization Using Crowdsourced Radio Maps

Authors: Zhaoguang Yi, Xiangyu Wen, Qiyue Xia, Peize Li, Francisco Zampella, Firas Alsehly, Chris Xiaoxuan Lu

Abstract: Indoor Positioning Systems (IPS) traditionally rely on odometry and building infrastructures like WiFi, often supplemented by building floor plans for increased accuracy. However, the limitation of floor plans in terms of availability and timeliness of updates challenges their wide applicability. In contrast, the proliferation of smartphones and WiFi-enabled robots has made crowdsourced radio maps… ▽ More Indoor Positioning Systems (IPS) traditionally rely on odometry and building infrastructures like WiFi, often supplemented by building floor plans for increased accuracy. However, the limitation of floor plans in terms of availability and timeliness of updates challenges their wide applicability. In contrast, the proliferation of smartphones and WiFi-enabled robots has made crowdsourced radio maps - databases pairing locations with their corresponding Received Signal Strengths (RSS) - increasingly accessible. These radio maps not only provide WiFi fingerprint-location pairs but encode movement regularities akin to the constraints imposed by floor plans. This work investigates the possibility of leveraging these radio maps as a substitute for floor plans in multimodal IPS. We introduce a new framework to address the challenges of radio map inaccuracies and sparse coverage. Our proposed system integrates an uncertainty-aware neural network model for WiFi localization and a bespoken Bayesian fusion technique for optimal fusion. Extensive evaluations on multiple real-world sites indicate a significant performance enhancement, with results showing ~ 25% improvement over the best baseline △ Less

Submitted 12 March, 2024; v1 submitted 17 November, 2023; originally announced November 2023.

Comments: 7 pages, 4 figures; ICRA'24 https://youtu.be/NTTKwJBFN5w

A Stochastic Online Forecast-and-Optimize Framework for Real-Time Energy Dispatch in Virtual Power Plants under Uncertainty

Authors: Wei Jiang, Zhongkai Yi, Li Wang, Hanwei Zhang, Jihai Zhang, Fangquan Lin, Cheng Yang

Abstract: Aggregating distributed energy resources in power systems significantly increases uncertainties, in particular caused by the fluctuation of renewable energy generation. This issue has driven the necessity of widely exploiting advanced predictive control techniques under uncertainty to ensure long-term economics and decarbonization. In this paper, we propose a real-time uncertainty-aware energy dis… ▽ More Aggregating distributed energy resources in power systems significantly increases uncertainties, in particular caused by the fluctuation of renewable energy generation. This issue has driven the necessity of widely exploiting advanced predictive control techniques under uncertainty to ensure long-term economics and decarbonization. In this paper, we propose a real-time uncertainty-aware energy dispatch framework, which is composed of two key elements: (i) A hybrid forecast-and-optimize sequential task, integrating deep learning-based forecasting and stochastic optimization, where these two stages are connected by the uncertainty estimation at multiple temporal resolutions; (ii) An efficient online data augmentation scheme, jointly involving model pre-training and online fine-tuning stages. In this way, the proposed framework is capable to rapidly adapt to the real-time data distribution, as well as to target on uncertainties caused by data drift, model discrepancy and environment perturbations in the control process, and finally to realize an optimal and robust dispatch solution. The proposed framework won the championship in CityLearn Challenge 2022, which provided an influential opportunity to investigate the potential of AI application in the energy domain. In addition, comprehensive experiments are conducted to interpret its effectiveness in the real-life scenario of smart building energy management. △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: Preprint. Accepted by CIKM 23

FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

Authors: Zhonghua Yi, Hao Shi, Kailun Yang, Qi Jiang, Yaozu Ye, Ze Wang, Huajian Ni, Kaiwei Wang

Abstract: Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-rele… ▽ More Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-relevant scenarios. To address these issues, we introduce a points-based modeling method that requires the model to learn key-point-related priors explicitly. Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point Control Loss (CPCL) function for diverse point-wise supervision; 2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition Control Encoder (CCE). CCE incorporates a Frame Feature Encoder (FFE) that extracts features from frames, a Condition Feature Encoder (CFE) that learns to control the feature extraction behavior of FFE from input masks containing information of key points, and fusion modules that transfer the controlling information between FFE and CFE. Our FocusFlow framework shows outstanding performance with up to +44.5% precision improvement on various key points such as ORB, SIFT, and even learning-based SiLK, along with exceptional scalability for most existing data-driven optical flow methods like PWC-Net, RAFT, and FlowFormer. Notably, FocusFlow yields competitive or superior performances rivaling the original models on the whole frame. The source code will be available at https://github.com/ZhonghuaYi/FocusFlow_official. △ Less

Submitted 22 September, 2023; v1 submitted 14 August, 2023; originally announced August 2023.

Comments: Accepted to IEEE Transactions on Intelligent Vehicles (T-IV). The source code of FocusFlow will be available at https://github.com/ZhonghuaYi/FocusFlow_official

Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers

Authors: Qi Jiang, Shaohua Gao, Yao Gao, Kailun Yang, Zhonghua Yi, Hao Shi, Lei Sun, Kaiwei Wang

Abstract: High-quality panoramic images with a Field of View (FoV) of 360-degree are essential for contemporary panoramic computer vision tasks. However, conventional imaging systems come with sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile and wearable applications where thin and portable, minimalist imaging systems are desired. In this paper, we propos… ▽ More High-quality panoramic images with a Field of View (FoV) of 360-degree are essential for contemporary panoramic computer vision tasks. However, conventional imaging systems come with sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile and wearable applications where thin and portable, minimalist imaging systems are desired. In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to address minimalist and high-quality panoramic imaging. With less than three spherical lenses, a Minimalist Panoramic Imaging Prototype (MPIP) is constructed based on the design of the Panoramic Annular Lens (PAL), but with low-quality imaging results due to aberrations and small image plane size. We propose two pipelines, i.e. Aberration Correction (AC) and Super-Resolution and Aberration Correction (SR&AC), to solve the image quality problems of MPIP, with imaging sensors of small and large pixel size, respectively. To provide a universal network for the two pipelines, we leverage the information from the Point Spread Function (PSF) of the optical system and design a PSF-aware Aberration-image Recovery Transformer (PART), in which the self-attention calculation and feature extraction are guided via PSF-aware mechanisms. We train PART on synthetic image pairs from simulation and put forward the PALHQ dataset to fill the gap of real-world high-quality PAL images for low-level vision. A comprehensive variety of experiments on synthetic and real-world benchmarks demonstrates the impressive imaging results of PCIE and the effectiveness of plug-and-play PSF-aware mechanisms. We further deliver heuristic experimental findings for minimalist and high-quality panoramic imaging. Our dataset and code will be available at https://github.com/zju-jiangqi/PCIE-PART. △ Less

Submitted 22 June, 2023; originally announced June 2023.

Comments: The dataset and code will be available at https://github.com/zju-jiangqi/PCIE-PART

Conservative Sparse Neural Network Embedded Frequency-Constrained Unit Commitment With Distributed Energy Resources

Authors: Linwei Sang, Yinliang Xu, Zhongkai Yi, Lun Yang, Huan Long, Hongbin Sun

Abstract: The increasing penetration of distributed energy resources (DERs) will decrease the rotational inertia of the power system and further degrade the system frequency stability. To address the above issues, this paper leverages the advanced neural network (NN) to learn the frequency dynamics and incorporates NN to facilitate system reliable operation. This paper proposes the conservative sparse neura… ▽ More The increasing penetration of distributed energy resources (DERs) will decrease the rotational inertia of the power system and further degrade the system frequency stability. To address the above issues, this paper leverages the advanced neural network (NN) to learn the frequency dynamics and incorporates NN to facilitate system reliable operation. This paper proposes the conservative sparse neural network (CSNN) embedded frequency-constrained unit commitment (FCUC) with converter-based DERs, including the learning and optimization stages. In the learning stage, it samples the inertia parameters, calculates the corresponding frequency, and characterizes the stability region of the sampled parameters using the convex hulls to ensure stability and avoid extrapolation. For conservativeness, the positive prediction error penalty is added to the loss function to prevent possible frequency requirement violation. For the sparsity, the NN topology pruning is employed to eliminate unnecessary connections for solving acceleration. In the optimization stage, the trained CSNN is transformed into mixed-integer linear constraints using the big-M method and then incorporated to establish the data-enhanced model. The case study verifies 1) the effectiveness of the proposed model in terms of high accuracy, fewer parameters, and significant solving acceleration; 2) the stable system operation against frequency violation under contingency. △ Less

Submitted 20 April, 2023; originally announced April 2023.

ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing

Authors: Bingchuan Li, Tianxiang Ma, Peng Zhang, Miao Hua, Wei Liu, Qian He, Zili Yi

Abstract: The StyleGAN family succeed in high-fidelity image generation and allow for flexible and plausible editing of generated images by manipulating the semantic-rich latent style space.However, projecting a real image into its latent space encounters an inherent trade-off between inversion quality and editability. Existing encoder-based or optimization-based StyleGAN inversion methods attempt to mitiga… ▽ More The StyleGAN family succeed in high-fidelity image generation and allow for flexible and plausible editing of generated images by manipulating the semantic-rich latent style space.However, projecting a real image into its latent space encounters an inherent trade-off between inversion quality and editability. Existing encoder-based or optimization-based StyleGAN inversion methods attempt to mitigate the trade-off but see limited performance. To fundamentally resolve this problem, we propose a novel two-phase framework by designating two separate networks to tackle editing and reconstruction respectively, instead of balancing the two. Specifically, in Phase I, a W-space-oriented StyleGAN inversion network is trained and used to perform image inversion and editing, which assures the editability but sacrifices reconstruction quality. In Phase II, a carefully designed rectifying network is utilized to rectify the inversion errors and perform ideal reconstruction. Experimental results show that our approach yields near-perfect reconstructions without sacrificing the editability, thus allowing accurate manipulation of real images. Further, we evaluate the performance of our rectifying network, and see great generalizability towards unseen manipulation types and out-of-domain images. △ Less

Submitted 30 January, 2023; originally announced January 2023.

Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

Authors: Andrey Ignatov, Radu Timofte, Shuai Liu, Chaoyu Feng, Furui Bai, Xiaotao Wang, Lei Lei, Ziyao Yi, Yan Xiang, Zibin Liu, Shaoqing Li, Keming Shi, Dehui Kong, Ke Xu, Minsu Kwon, Yaqi Wu, Jiesi Zheng, Zhihao Fan, Xun Wu, Feng Zhang, Albert No, Minhyeok Cho, Zewen Chen, Xiaze Zhang, Ran Li , et al. (13 additional authors not shown)

Abstract: The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. Th… ▽ More The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Calibrationless Reconstruction of Uniformly-Undersampled Multi-Channel MR Data with Deep Learning Estimated ESPIRiT Maps

Authors: Junhao Zhang, Zheyuan Yi, Yujiao Zhao, Linfang Xiao, Jiahao Hu, Christopher Man, Vick Lau, Shi Su, Fei Chen, Alex T. L. Leong, Ed X. Wu

Abstract: Purpose: To develop a truly calibrationless reconstruction method that derives ESPIRiT maps from uniformly-undersampled multi-channel MR data by deep learning. Methods: ESPIRiT, one commonly used parallel imaging reconstruction technique, forms the images from undersampled MR k-space data using ESPIRiT maps that effectively represents coil sensitivity information. Accurate ESPIRiT map estimation r… ▽ More Purpose: To develop a truly calibrationless reconstruction method that derives ESPIRiT maps from uniformly-undersampled multi-channel MR data by deep learning. Methods: ESPIRiT, one commonly used parallel imaging reconstruction technique, forms the images from undersampled MR k-space data using ESPIRiT maps that effectively represents coil sensitivity information. Accurate ESPIRiT map estimation requires quality coil sensitivity calibration or autocalibration data. We present a U-Net based deep learning model to estimate the multi-channel ESPIRiT maps directly from uniformly-undersampled multi-channel multi-slice MR data. The model is trained using fully-sampled multi-slice axial brain datasets from the same MR receiving coil system. To utilize subject-coil geometric parameters available for each dataset, the training imposes a hybrid loss on ESPIRiT maps at the original locations as well as their corresponding locations within the standard reference multi-slice axial stack. The performance of the approach was evaluated using publicly available T1-weighed brain and cardiac data. Results: The proposed model robustly predicted multi-channel ESPIRiT maps from uniformly-undersampled k-space data. They were highly comparable to the reference ESPIRiT maps directly computed from 24 consecutive central k-space lines. Further, they led to excellent ESPIRiT reconstruction performance even at high acceleration, exhibiting a similar level of errors and artifacts to that by using reference ESPIRiT maps. Conclusion: A new deep learning approach is developed to estimate ESPIRiT maps directly from uniformly-undersampled MR data. It presents a general strategy for calibrationless parallel imaging reconstruction through learning from coil and protocol specific data. △ Less

Submitted 27 October, 2022; v1 submitted 26 October, 2022; originally announced October 2022.

A Learning-Based 3D EIT Image Reconstruction Method

Authors: Zhaoguang Yi, Zhou Chen, Yunjie Yang

Abstract: Deep learning has been widely employed to solve the Electrical Impedance Tomography (EIT) image reconstruction problem. Most existing physical model-based and learning-based approaches focus on 2D EIT image reconstruction. However, when they are directly extended to the 3D domain, the reconstruction performance in terms of image quality and noise robustness is hardly guaranteed mainly due to the s… ▽ More Deep learning has been widely employed to solve the Electrical Impedance Tomography (EIT) image reconstruction problem. Most existing physical model-based and learning-based approaches focus on 2D EIT image reconstruction. However, when they are directly extended to the 3D domain, the reconstruction performance in terms of image quality and noise robustness is hardly guaranteed mainly due to the significant increase in dimensionality. This paper presents a learning-based approach for 3D EIT image reconstruction, which is named Transposed convolution with Neurons Network (TN-Net). Simulation and experimental results show the superior performance and generalization ability of TN-Net compared with prevailing 3D EIT image reconstruction algorithms. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Journal ref: Proceedings of the International Conference of Bioelectromagnetism, Electrical Bioimpedance, and Electrical Impedance Tomography. June 28 to July 1, 2022 Kyung Hee University, Seoul, Korea

RestainNet: a self-supervised digital re-stainer for stain normalization

Authors: Bingchao Zhao, Jiatai Lin, Changhong Liang, Zongjian Yi, Xin Chen, Bingbing Li, Weihao Qiu, Danyi Li, Li Liang, Chu Han, Zaiyi Liu

Abstract: Color inconsistency is an inevitable challenge in computational pathology, which generally happens because of stain intensity variations or sections scanned by different scanners. It harms the pathological image analysis methods, especially the learning-based models. A series of approaches have been proposed for stain normalization. However, most of them are lack flexibility in practice. In this p… ▽ More Color inconsistency is an inevitable challenge in computational pathology, which generally happens because of stain intensity variations or sections scanned by different scanners. It harms the pathological image analysis methods, especially the learning-based models. A series of approaches have been proposed for stain normalization. However, most of them are lack flexibility in practice. In this paper, we formulated stain normalization as a digital re-staining process and proposed a self-supervised learning model, which is called RestainNet. Our network is regarded as a digital restainer which learns how to re-stain an unstained (grayscale) image. Two digital stains, Hematoxylin (H) and Eosin (E) were extracted from the original image by Beer-Lambert's Law. We proposed a staining loss to maintain the correctness of stain intensity during the restaining process. Thanks to the self-supervised nature, paired training samples are no longer necessary, which demonstrates great flexibility in practical usage. Our RestainNet outperforms existing approaches and achieves state-of-the-art performance with regard to color correctness and structure preservation. We further conducted experiments on the segmentation and classification tasks and the proposed RestainNet achieved outstanding performance compared with SOTA methods. The self-supervised design allows the network to learn any staining style with no extra effort. △ Less

Submitted 28 February, 2022; originally announced February 2022.

Imaging through scattering media via spatial-temporal encoded pattern illumination

Authors: Xingchen Zhao, Xiaoyu Nie, Zhenhuan Yi, Tao Peng, Marlan O. Scully

Abstract: Optical imaging through scattering media is a long-standing challenge. Although many approaches have been developed to focus light or image objects through scattering media, they are either invasive, restricted to stationary or slowly-moving media, or require high-resolution cameras and complex algorithms to retrieve the images. Here we introduce a computational imaging technique that can overcome… ▽ More Optical imaging through scattering media is a long-standing challenge. Although many approaches have been developed to focus light or image objects through scattering media, they are either invasive, restricted to stationary or slowly-moving media, or require high-resolution cameras and complex algorithms to retrieve the images. Here we introduce a computational imaging technique that can overcome these restrictions by exploiting spatial-temporal encoded patterns (STEP). We present non-invasive imaging through scattering media with a single-pixel photodetector. We show that the method is insensitive to the motions of media. We further demonstrate that our image reconstruction algorithm is much more efficient than correlation-based algorithms for single-pixel imaging, which may allow fast imaging in currently unreachable scenarios. △ Less

Submitted 25 December, 2021; originally announced December 2021.

Comments: 7 pages, 4 figures

Attention-Based Sensor Fusion for Human Activity Recognition Using IMU Signals

Authors: Wen** Tao, Haodong Chen, Md Moniruzzaman, Ming C. Leu, Zhaozheng Yi, Ruwen Qin

Abstract: Human Activity Recognition (HAR) using wearable devices such as smart watches embedded with Inertial Measurement Unit (IMU) sensors has various applications relevant to our daily life, such as workout tracking and health monitoring. In this paper, we propose a novel attention-based approach to human activity recognition using multiple IMU sensors worn at different body locations. Firstly, a sensor… ▽ More Human Activity Recognition (HAR) using wearable devices such as smart watches embedded with Inertial Measurement Unit (IMU) sensors has various applications relevant to our daily life, such as workout tracking and health monitoring. In this paper, we propose a novel attention-based approach to human activity recognition using multiple IMU sensors worn at different body locations. Firstly, a sensor-wise feature extraction module is designed to extract the most discriminative features from individual sensors with Convolutional Neural Networks (CNNs). Secondly, an attention-based fusion mechanism is developed to learn the importance of sensors at different body locations and to generate an attentive feature representation. Finally, an inter-sensor feature extraction module is applied to learn the inter-sensor correlations, which are connected to a classifier to output the predicted classes of activities. The proposed approach is evaluated using five public datasets and it outperforms state-of-the-art methods on a wide variety of activity categories. △ Less

Submitted 20 December, 2021; originally announced December 2021.

RegNet: Self-Regulated Network for Image Classification

Authors: **g Xu, Yu Pan, Xinglin Pan, Steven Hoi, Zhang Yi, Zenglin Xu

Abstract: The ResNet and its variants have achieved remarkable successes in various computer vision tasks. Despite its success in making gradient flow through building blocks, the simple shortcut connection mechanism limits the ability of re-exploring new potentially complementary features due to the additive function. To address this issue, in this paper, we propose to introduce a regulator module as a mem… ▽ More The ResNet and its variants have achieved remarkable successes in various computer vision tasks. Despite its success in making gradient flow through building blocks, the simple shortcut connection mechanism limits the ability of re-exploring new potentially complementary features due to the additive function. To address this issue, in this paper, we propose to introduce a regulator module as a memory mechanism to extract complementary features, which are further fed to the ResNet. In particular, the regulator module is composed of convolutional RNNs (e.g., Convolutional LSTMs or Convolutional GRUs), which are shown to be good at extracting Spatio-temporal information. We named the new regulated networks as RegNet. The regulator module can be easily implemented and appended to any ResNet architecture. We also apply the regulator module for improving the Squeeze-and-Excitation ResNet to show the generalization ability of our method. Experimental results on three image classification datasets have demonstrated the promising performance of the proposed architecture compared with the standard ResNet, SE-ResNet, and other state-of-the-art architectures. △ Less

Submitted 3 January, 2021; originally announced January 2021.

Comments: 6 pages, 4 figures

Animating Through War**: an Efficient Method for High-Quality Facial Expression Animation

Authors: Zili Yi, Qiang Tang, Vishnu Sanjay Ramiya Srinivasan, Zhan Xu

Abstract: Advances in deep neural networks have considerably improved the art of animating a still image without operating in 3D domain. Whereas, prior arts can only animate small images (typically no larger than 512x512) due to memory limitations, difficulty of training and lack of high-resolution (HD) training datasets, which significantly reduce their potential for applications in movie production and in… ▽ More Advances in deep neural networks have considerably improved the art of animating a still image without operating in 3D domain. Whereas, prior arts can only animate small images (typically no larger than 512x512) due to memory limitations, difficulty of training and lack of high-resolution (HD) training datasets, which significantly reduce their potential for applications in movie production and interactive systems. Motivated by the idea that HD images can be generated by adding high-frequency residuals to low-resolution results produced by a neural network, we propose a novel framework known as Animating Through War** (ATW) to enable efficient animation of HD images. Specifically, the proposed framework consists of two modules, a novel two-stage neural-network generator and a novel post-processing module known as Animating Through War** (ATW). It only requires the generator to be trained on small images and can do inference on an image of any size. During inference, an HD input image is decomposed into a low-resolution component(128x128) and its corresponding high-frequency residuals. The generator predicts the low-resolution result as well as the motion field that warps the input face to the desired status (e.g., expressions categories or action units). Finally, the ResWarp module warps the residuals based on the motion field and adding the warped residuals to generates the final HD results from the naively up-sampled low-resolution results. Experiments show the effectiveness and efficiency of our method in generating high-resolution animations. Our proposed framework successfully animates a 4K facial image, which has never been achieved by prior neural models. In addition, our method generally guarantee the temporal coherency of the generated animations. Source codes will be made publicly available. △ Less

Submitted 1 August, 2020; originally announced August 2020.

Comments: 18 pages, 13 figures, Accepted to ACM Multimedia 2020

ACM Class: I.3.3; J.6; I.2.10

Distributed Frequency Emergency Control with Coordinated Edge Intelligence

Authors: Yingmeng Xiang, Zhehan Yi, Xiao Lu, Zhe Yu, Di Shi, Chunlei Xu, Xueming Li, Zhiwei Wang

Abstract: Develo** effective strategies to rapidly support grid frequency while minimizing loss in case of severe contingencies is an important requirement in power systems. While distributed responsive load demands are commonly adopted for frequency regulation, it is difficult to achieve both rapid response and global accuracy in a practical and cost-effective manner. In this paper, the cyber-physical de… ▽ More Develo** effective strategies to rapidly support grid frequency while minimizing loss in case of severe contingencies is an important requirement in power systems. While distributed responsive load demands are commonly adopted for frequency regulation, it is difficult to achieve both rapid response and global accuracy in a practical and cost-effective manner. In this paper, the cyber-physical design of an Internet-of-Things (IoT) enabled system, called Grid Sense, is presented. Grid Sense utilizes a large number of distributed appliances for frequency emergency support. It features a local power loss $ΔP$ estimation approach for frequency emergency control based on coordinated edge intelligence. The specifically designed smart outlets of Grid Sense detect the frequency disturbance event locally using the parameters sent from the control center to estimate active power loss in the system and to make rapid and accurate switching decisions soon after a severe contingency. Based on a modified IEEE 24-bus system, numerical simulations and hardware experiments are conducted to demonstrate the frequency support performance of Grid Sense in the aspects of accuracy and speed. It is shown that Grid Sense equipped with its local $ΔP$-estimation frequency control approach can accurately and rapidly prevent the drop of frequency after a major power loss. △ Less

Submitted 24 February, 2020; originally announced February 2020.

Nonlinear Covariance Control via Differential Dynamic Programming

Authors: Zeji Yi, Zhefeng Cao, Evangelos Theodorou, Yongxin Chen

Abstract: We consider covariance control problems for nonlinear stochastic systems. Our objective is to find an optimal control strategy to steer the state from an initial distribution to a terminal one with specified mean and covariance. This problem is considerably more complicated than previous studies on covariance control for linear systems. We leverage a widely used technique - differential dynamic pr… ▽ More We consider covariance control problems for nonlinear stochastic systems. Our objective is to find an optimal control strategy to steer the state from an initial distribution to a terminal one with specified mean and covariance. This problem is considerably more complicated than previous studies on covariance control for linear systems. We leverage a widely used technique - differential dynamic programming - in nonlinear optimal control to achieve our goal. In particular, we adopt the stochastic differential dynamic programming framework to handle the stochastic dynamics. Additionally, to enforce the terminal statistical constraints, we construct a Lagrangian and apply a primal-dual type algorithm. Several examples are presented to demonstrate the effectiveness of our framework. △ Less

Submitted 20 November, 2019; originally announced November 2019.

Comments: 7 pages, 5 figures

A Rprop-Neural-Network-Based PV Maximum Power Point Tracking Algorithm with Short-Circuit Current Limitation

Authors: Yao Cui, Zhehan Yi, Jiajun Duan, Di Shi, Zhiwei Wang

Abstract: This paper proposes a resilient-backpropagation-neural-network-(Rprop-NN) based algorithm for Photovoltaic (PV) maximum power point tracking (MPPT). A supervision mechanism is proposed to calibrate the Rprop-NN-MPPT reference and limit short-circuit current caused by incorrect prediction. Conventional MPPT algorithms (e.g., perturb and observe (P&O), hill climbing, and incremental conductance (Inc… ▽ More This paper proposes a resilient-backpropagation-neural-network-(Rprop-NN) based algorithm for Photovoltaic (PV) maximum power point tracking (MPPT). A supervision mechanism is proposed to calibrate the Rprop-NN-MPPT reference and limit short-circuit current caused by incorrect prediction. Conventional MPPT algorithms (e.g., perturb and observe (P&O), hill climbing, and incremental conductance (Inc-Cond) etc.) are trial-and-error-based, which may result in steady-state oscillations and loss of tracking direction under fast-changing ambient environment. In addition, partial shading is also a challenge due to the difficulty of finding the global maximum power point on a multi-peak characteristic curve. As an attempt to address the aforementioned issues, a novel Rprop-NN MPPT algorithm is developed and elaborated in this work. Multiple case studies are carried out to verify the effectiveness of the proposed algorithm. △ Less

Submitted 6 December, 2018; v1 submitted 29 November, 2018; originally announced November 2018.

Comments: 2019 IEEE ISGT NA

A Neural-Network-Based Optimal Control of Ultra-Capacitors with System Uncertainties

Authors: Jiajun Duan, Zhehan Yi, Di Shi, Hao Xu, Zhiwei Wang

Abstract: In this paper, a neural-network (NN)-based online optimal control method (NN-OPT) is proposed for ultra-capacitors (UCs) energy storage system (ESS) in hybrid AC/DC microgrids involving multiple distributed generations (e.g., Photovoltaic (PV) system, battery storage, diesel generator). Conventional control strategies usually produce large disturbances to buses during charging and discharging (C&D… ▽ More In this paper, a neural-network (NN)-based online optimal control method (NN-OPT) is proposed for ultra-capacitors (UCs) energy storage system (ESS) in hybrid AC/DC microgrids involving multiple distributed generations (e.g., Photovoltaic (PV) system, battery storage, diesel generator). Conventional control strategies usually produce large disturbances to buses during charging and discharging (C&D) processes of UCs, which significantly degrades the power quality and system performance, especially under fast C&D modes. Therefore, the optimal control theory is adopted to optimize the C&D profile as well as to suppress the disturbances caused by UCs implementation. Specifically, an NN-based intelligent algorithm is developed to learn the optimal control policy for bidirectional-converter-interfaced UCs. The inaccuracies of system modeling are also considered in the control design. Since the designed NN-OPT method is decentralized that only requires the local measurements, plug & play of UCs can be easily realized with minimal communication efforts. In addition, the PV system is under the maximum power point tracking (MPPT) control to extract the maximum benefit. Both islanded and grid-tied modes are considered during the controller design. Extensive case studies have been conducted to evaluate the effectiveness of the proposed method. △ Less

Submitted 15 April, 2019; v1 submitted 29 November, 2018; originally announced November 2018.

Comments: IEEE ISGT NA 2019

Small-signal Stability Analysis and Performance Evaluation of Microgrids under Distributed Control

Authors: Yimajian Yan, Di Shi, Desong Bian, Bibin Huang, Zhehan Yi, Zhiwei Wang

Abstract: Distributed control, as a potential solution to decreasing communication demands in microgrids, has drawn much attention in recent years. Advantages of distributed control have been extensively discussed, while its impacts on microgrid performance and stability, especially in the case of communication latency, have not been explicitly studied or fully understood yet. This paper addresses this gap… ▽ More Distributed control, as a potential solution to decreasing communication demands in microgrids, has drawn much attention in recent years. Advantages of distributed control have been extensively discussed, while its impacts on microgrid performance and stability, especially in the case of communication latency, have not been explicitly studied or fully understood yet. This paper addresses this gap by proposing a generalized theoretical framework for small-signal stability analysis and performance evaluation for microgrids using distributed control. The proposed framework synthesizes generator and load frequency-domain characteristics, primary and secondary control loops, as well as the communication latency into a frequency-domain representation which is further evaluated by the generalized Nyquist theorem. In addition, various parameters and their impacts on microgrid dynamic performance are investigated and summarized into guidelines to help better design the system. Case studies demonstrate the effectiveness of the proposed approach. △ Less

Submitted 7 September, 2018; v1 submitted 4 September, 2018; originally announced September 2018.

Model Predictive Control of H5 Inverter for Transformerless PV Systems with Maximum Power Point Tracking and Leakage Current Reduction

Authors: Abdulrahman J. Babqi, Zhehan Yi, Di Shi, Xiaoying Zhao

Abstract: Transformerless grid-connected solar photovoltaic (PV) systems have given rise to more research and commercial interests due to their multiple merits, e.g., low leakage current and small size. In this paper, a model-predictive-control (MPC)-based strategy for controlling transformerless H5 inverter for single-phase PV distributed generation system is proposed. The method further reduces the PV lea… ▽ More Transformerless grid-connected solar photovoltaic (PV) systems have given rise to more research and commercial interests due to their multiple merits, e.g., low leakage current and small size. In this paper, a model-predictive-control (MPC)-based strategy for controlling transformerless H5 inverter for single-phase PV distributed generation system is proposed. The method further reduces the PV leakage current in a cost-effective and safe manner and it shows a satisfactory fault-ride-through capability. Moreover, for the first of its kind, PV maximum power point tracking is implemented in the single-stage H5 inverter using MPC-based controllers. Various case studies are carried out, which provide the result comparisons between the proposed and conventional control methods and verify the promising performance of the proposed method. △ Less

Submitted 26 July, 2018; originally announced July 2018.

Comments: This work has been accepted by the 44th Annual Conference of the IEEE Industrial Electronics Society (IECON 2018). This is a preprint. DOI is to be added

Finite-Control-Set Model Predictive Control (FCS-MPC) for Islanded Hybrid Microgrids

Authors: Zhehan Yi, Abdulrahman J. Babqi, Yishen Wang, Di Shi, Amir H. Etemadi, Zhiwei Wang, Bibin Huang

Abstract: Microgrids consisting of multiple distributed energy resources (DERs) provide a promising solution to integrate renewable energies, e.g., solar photovoltaic (PV) systems. Hybrid AC/DC microgrids leverage the merits of both AC and DC power systems. In this paper, a control strategy for islanded multi-bus hybrid microgrids is proposed based on the Finite-Control-Set Model Predictive Control (FCS-MPC… ▽ More Microgrids consisting of multiple distributed energy resources (DERs) provide a promising solution to integrate renewable energies, e.g., solar photovoltaic (PV) systems. Hybrid AC/DC microgrids leverage the merits of both AC and DC power systems. In this paper, a control strategy for islanded multi-bus hybrid microgrids is proposed based on the Finite-Control-Set Model Predictive Control (FCS-MPC) technologies. The control loops are expedited by predicting the future states and determining the optimal control action before switching signals are sent. The proposed algorithm eliminates the needs of PI, PWM, and droop components, and offers 1) accurate PV maximum power point tracking (MPPT) and battery charging/discharging control, 2) DC and multiple AC bus voltage/frequency regulation, 3) a precise power sharing scheme among DERs without voltage or frequency deviation, and 4) a unified MPC design flow for hybrid microgrids. Multiple case studies are carried out, which verify the satisfactory performance of the proposed method. △ Less

Submitted 12 February, 2018; originally announced February 2018.

Comments: This paper has been accepted by the 2018 IEEE PES General Meeting

An Extended Kalman Filter Enhanced Hilbert-Huang Transform in Oscillation Detection

Authors: Zhe Yu, Di Shi, Haifeng Li, Yishen Wang, Zhehan Yi, Zhiwei Wang

Abstract: Hilbert-Huang transform (HHT) has drawn great attention in power system analysis due to its capability to deal with dynamic signal and provide instantaneous characteristics such as frequency, dam**, and amplitudes. However, its shortcomings, including mode mixing and end effects, are as significant as its advantages. A preliminary result of an extended Kalman filter (EKF) method to enhance HHT a… ▽ More Hilbert-Huang transform (HHT) has drawn great attention in power system analysis due to its capability to deal with dynamic signal and provide instantaneous characteristics such as frequency, dam**, and amplitudes. However, its shortcomings, including mode mixing and end effects, are as significant as its advantages. A preliminary result of an extended Kalman filter (EKF) method to enhance HHT and hopefully to overcome these disadvantages is presented in this paper. The proposal first removes dynamic DC components in signals using empirical mode decomposition. Then an EKF model is applied to extract instant coefficients. Numerical results using simulated and real-world low-frequency oscillation data suggest the proposal can help to overcome the mode mixing and end effects with a properly chosen number of modes. △ Less

Submitted 8 November, 2017; originally announced November 2017.

Comments: 5 pages, 2 figures. Submitted to 2018 IEEE PES General Meeting. arXiv admin note: text overlap with arXiv:1706.05355

A Centralized Power Control and Management Method for Grid-Connected Photovoltaic (PV)-Battery Systems

Authors: Zhehan Yi, Wanxin Dong, Amir H. Etemadi

Abstract: Distributed Generation (DG) is an effective way of integrating renewable energy sources to conventional power grid, which improves the reliability and efficiency of power systems. Photovoltaic (PV) systems are ideal DGs thanks to their attractive benefits, such as availability of solar energy and low installation costs. Battery groups are used in PV systems to balance the power flows and eliminate… ▽ More Distributed Generation (DG) is an effective way of integrating renewable energy sources to conventional power grid, which improves the reliability and efficiency of power systems. Photovoltaic (PV) systems are ideal DGs thanks to their attractive benefits, such as availability of solar energy and low installation costs. Battery groups are used in PV systems to balance the power flows and eliminate power fluctuations due to change of operating condition, e.g., irradiance and temperature variation. In an attempt to effectively manage the power flows, this paper presents a novel power control and management system for grid-connected PV-Battery systems. The proposed system realizes the maximum power point tracking (MPPT) of the PV panels, stabilization of the DC bus voltage for load plug-and-play access, balance among the power flows, and quick response of both active and reactive power demands. △ Less

Submitted 26 September, 2017; originally announced September 2017.