Showing 1–26 of 26 results for author: Meng, F

Knowledge-driven Subspace Fusion and Gradient Coordination for Multi-modal Learning

Authors: Yupei Zhang, Xiaofei Wang, Fangliangzi Meng, ** Tang, Chao Li

Abstract: Multi-modal learning plays a crucial role in cancer diagnosis and prognosis. Current deep learning based multi-modal approaches are often limited by their abilities to model the complex correlations between genomics and histology data, addressing the intrinsic complexity of tumour ecosystem where both tumour and microenvironment contribute to malignancy. We propose a biologically interpretative an… ▽ More Multi-modal learning plays a crucial role in cancer diagnosis and prognosis. Current deep learning based multi-modal approaches are often limited by their abilities to model the complex correlations between genomics and histology data, addressing the intrinsic complexity of tumour ecosystem where both tumour and microenvironment contribute to malignancy. We propose a biologically interpretative and robust multi-modal learning framework to efficiently integrate histology images and genomics by decomposing the feature subspace of histology images and genomics, reflecting distinct tumour and microenvironment features. To enhance cross-modal interactions, we design a knowledge-driven subspace fusion scheme, consisting of a cross-modal deformable attention module and a gene-guided consistency strategy. Additionally, in pursuit of dynamically optimizing the subspace knowledge, we further propose a novel gradient coordination learning strategy. Extensive experiments demonstrate the effectiveness of the proposed method, outperforming state-of-the-art techniques in three downstream tasks of glioma diagnosis, tumour grading, and survival analysis. Our code is available at https://github.com/helenypzhang/Subspace-Multimodal-Learning. △ Less

Submitted 20 June, 2024; originally announced June 2024.

On the Adversarial Robustness of Learning-based Image Compression Against Rate-Distortion Attacks

Authors: Chenhao Wu, Qingbo Wu, Haoran Wei, Shuai Chen, Lei Wang, King Ngi Ngan, Fanman Meng, Hongliang Li

Abstract: Despite demonstrating superior rate-distortion (RD) performance, learning-based image compression (LIC) algorithms have been found to be vulnerable to malicious perturbations in recent studies. Adversarial samples in these studies are designed to attack only one dimension of either bitrate or distortion, targeting a submodel with a specific compression ratio. However, adversaries in real-world sce… ▽ More Despite demonstrating superior rate-distortion (RD) performance, learning-based image compression (LIC) algorithms have been found to be vulnerable to malicious perturbations in recent studies. Adversarial samples in these studies are designed to attack only one dimension of either bitrate or distortion, targeting a submodel with a specific compression ratio. However, adversaries in real-world scenarios are neither confined to singular dimensional attacks nor always have control over compression ratios. This variability highlights the inadequacy of existing research in comprehensively assessing the adversarial robustness of LIC algorithms in practical applications. To tackle this issue, this paper presents two joint rate-distortion attack paradigms at both submodel and algorithm levels, i.e., Specific-ratio Rate-Distortion Attack (SRDA) and Agnostic-ratio Rate-Distortion Attack (ARDA). Additionally, a suite of multi-granularity assessment tools is introduced to evaluate the attack results from various perspectives. On this basis, extensive experiments on eight prominent LIC algorithms are conducted to offer a thorough analysis of their inherent vulnerabilities. Furthermore, we explore the efficacy of two defense techniques in improving the performance under joint rate-distortion attacks. The findings from these experiments can provide a valuable reference for the development of compression algorithms with enhanced adversarial robustness. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Destination-Constrained Linear Dynamical System Modeling in Set-Valued Frameworks

Authors: Xiaowei Yang, Haiqi Liu, Fanqin Meng, Xiao**g Shen

Abstract: Directional motion towards a specified destination is a common occurrence in physical processes and human societal activities. Utilizing this prior information can significantly improve the control and predictive performance of system models. This paper primarily focuses on reconstructing linear dynamic system models based on destination constraints in the set-valued framework. We treat destinatio… ▽ More Directional motion towards a specified destination is a common occurrence in physical processes and human societal activities. Utilizing this prior information can significantly improve the control and predictive performance of system models. This paper primarily focuses on reconstructing linear dynamic system models based on destination constraints in the set-valued framework. We treat destination constraints as inherent information in the state evolution process and employ convex optimization techniques to construct a coherent and robust state model. This refined model effectively captures the impact of destination constraints on the state evolution at each time step. Furthermore, we design an optimal weight matrix for the reconstructed model to ensure smoother and more natural trajectories of state evolution. We also analyze the theoretical guarantee of optimality for this weight matrix and the properties of the reconstructed model. Finally, simulation experiments verify that the reconstructed model has significant advantages over the unconstrained and unoptimized weighted models and constrains the evolution of state trajectories with different starting and ending points. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: 15 pages, 11 figures

Finding the Missing Data: A BERT-inspired Approach Against Package Loss in Wireless Sensing

Authors: Zijian Zhao, Tingwei Chen, Fanyi Meng, Hang Li, Xiaoyang Li, Guangxu Zhu

Abstract: Despite the development of various deep learning methods for Wi-Fi sensing, package loss often results in noncontinuous estimation of the Channel State Information (CSI), which negatively impacts the performance of the learning models. To overcome this challenge, we propose a deep learning model based on Bidirectional Encoder Representations from Transformers (BERT) for CSI recovery, named CSI-BER… ▽ More Despite the development of various deep learning methods for Wi-Fi sensing, package loss often results in noncontinuous estimation of the Channel State Information (CSI), which negatively impacts the performance of the learning models. To overcome this challenge, we propose a deep learning model based on Bidirectional Encoder Representations from Transformers (BERT) for CSI recovery, named CSI-BERT. CSI-BERT can be trained in an self-supervised manner on the target dataset without the need for additional data. Furthermore, unlike traditional interpolation methods that focus on one subcarrier at a time, CSI-BERT captures the sequential relationships across different subcarriers. Experimental results demonstrate that CSI-BERT achieves lower error rates and faster speed compared to traditional interpolation methods, even when facing with high loss rates. Moreover, by harnessing the recovered CSI obtained from CSI-BERT, other deep learning models like Residual Network and Recurrent Neural Network can achieve an average increase in accuracy of approximately 15\% in Wi-Fi sensing tasks. The collected dataset WiGesture and code for our model are publicly available at https://github.com/RS2002/CSI-BERT. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: 6 pages, accepted by IEEE INFOCOM Deepwireless Workshop 2024

Enhancing Adversarial Training with Prior Knowledge Distillation for Robust Image Compression

Authors: Zhi Cao, Youneng Bao, Fanyang Meng, Chao Li, Wen Tan, Genhong Wang, Yongsheng Liang

Abstract: Deep neural network-based image compression (NIC) has achieved excellent performance, but NIC method models have been shown to be susceptible to backdoor attacks. Adversarial training has been validated in image compression models as a common method to enhance model robustness. However, the improvement effect of adversarial training on model robustness is limited. In this paper, we propose a prior… ▽ More Deep neural network-based image compression (NIC) has achieved excellent performance, but NIC method models have been shown to be susceptible to backdoor attacks. Adversarial training has been validated in image compression models as a common method to enhance model robustness. However, the improvement effect of adversarial training on model robustness is limited. In this paper, we propose a prior knowledge-guided adversarial training framework for image compression models. Specifically, first, we propose a gradient regularization constraint for training robust teacher models. Subsequently, we design a knowledge distillation based strategy to generate a priori knowledge from the teacher model to the student model for guiding adversarial training. Experimental results show that our method improves the reconstruction quality by about 9dB when the Kodak dataset is elected as the backdoor attack object for psnr attack. Compared with Ma2023, our method has a 5dB higher PSNR output at high bitrate points. △ Less

Submitted 15 March, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Entropy-based Probing Beam Selection and Beam Prediction via Deep Learning

Authors: Fan Meng, Cheng Zhang, Yongming Huang, Zhilei Zhang, Xiaoyu Bai, Zhaohua Lu

Abstract: Hierarchical beam search in mmWave communications incurs substantial training overhead, necessitating deep learning-enabled beam predictions to effectively leverage channel priors and mitigate this overhead. In this study, we introduce a comprehensive probabilistic model of power distribution in beamspace, and formulate the joint optimization problem of probing beam selection and probabilistic bea… ▽ More Hierarchical beam search in mmWave communications incurs substantial training overhead, necessitating deep learning-enabled beam predictions to effectively leverage channel priors and mitigate this overhead. In this study, we introduce a comprehensive probabilistic model of power distribution in beamspace, and formulate the joint optimization problem of probing beam selection and probabilistic beam prediction as an entropy minimization problem. Then, we propose a greedy scheme to iteratively and alternately solve this problem, where a transformer-based beam predictor is trained to estimate the conditional power distribution based on the probing beams and user location within each iteration, and the trained predictor selects an unmeasured beam that minimizes the entropy of remaining beams. To further reduce the number of interactions and the computational complexity of the iterative scheme, we propose a two-stage probing beam selection scheme. Firstly, probing beams are selected from a location-specific codebook designed by an entropy-based criterion, and predictions are made with corresponding feedback. Secondly, the optimal beam is identified using additional probing beams with the highest predicted power values. Simulation results demonstrate the superiority of the proposed schemes compared to hierarchical beam search and beam prediction with uniform probing beams. △ Less

Submitted 3 January, 2024; originally announced January 2024.

Joint Detection Algorithm for Multiple Cognitive Users in Spectrum Sensing

Authors: Fanfei Meng, Yuxin Wang, Lele Zhang, Yingxin Zhao

Abstract: Spectrum sensing technology is a crucial aspect of modern communication technology, serving as one of the essential techniques for efficiently utilizing scarce information resources in tight frequency bands. This paper first introduces three common logical circuit decision criteria in hard decisions and analyzes their decision rigor. Building upon hard decisions, the paper further introduces a met… ▽ More Spectrum sensing technology is a crucial aspect of modern communication technology, serving as one of the essential techniques for efficiently utilizing scarce information resources in tight frequency bands. This paper first introduces three common logical circuit decision criteria in hard decisions and analyzes their decision rigor. Building upon hard decisions, the paper further introduces a method for multi-user spectrum sensing based on soft decisions. Then the paper simulates the false alarm probability and detection probability curves corresponding to the three criteria. The simulated results of multi-user collaborative sensing indicate that the simulation process significantly reduces false alarm probability and enhances detection probability. This approach effectively detects spectrum resources unoccupied during idle periods, leveraging the concept of time-division multiplexing and rationalizing the redistribution of information resources. The entire computation process relies on the calculation principles of power spectral density in communication theory, involving threshold decision detection for noise power and the sum of noise and signal power. It provides a secondary decision detection, reflecting the perceptual decision performance of logical detection methods with relative accuracy. △ Less

Submitted 1 December, 2023; v1 submitted 30 November, 2023; originally announced November 2023.

Comments: https://aei.ewapublishing.org/article.html?pk=e24c40d220434209ae2fe2e984bcf2c2

Journal ref: Advances in Engineering Innovation, Vol. 4, 16-25, Published 27 November 2023

Learning with Noisy Low-Cost MOS for Image Quality Assessment via Dual-Bias Calibration

Authors: Lei Wang, Qingbo Wu, Desen Yuan, King Ngi Ngan, Hongliang Li, Fanman Meng, Linfeng Xu

Abstract: Learning based image quality assessment (IQA) models have obtained impressive performance with the help of reliable subjective quality labels, where mean opinion score (MOS) is the most popular choice. However, in view of the subjective bias of individual annotators, the labor-abundant MOS (LA-MOS) typically requires a large collection of opinion scores from multiple annotators for each image, whi… ▽ More Learning based image quality assessment (IQA) models have obtained impressive performance with the help of reliable subjective quality labels, where mean opinion score (MOS) is the most popular choice. However, in view of the subjective bias of individual annotators, the labor-abundant MOS (LA-MOS) typically requires a large collection of opinion scores from multiple annotators for each image, which significantly increases the learning cost. In this paper, we aim to learn robust IQA models from low-cost MOS (LC-MOS), which only requires very few opinion scores or even a single opinion score for each image. More specifically, we consider the LC-MOS as the noisy observation of LA-MOS and enforce the IQA model learned from LC-MOS to approach the unbiased estimation of LA-MOS. In this way, we represent the subjective bias between LC-MOS and LA-MOS, and the model bias between IQA predictions learned from LC-MOS and LA-MOS (i.e., dual-bias) as two latent variables with unknown parameters. By means of the expectation-maximization based alternating optimization, we can jointly estimate the parameters of the dual-bias, which suppresses the misleading of LC-MOS via a gated dual-bias calibration (GDBC) module. To the best of our knowledge, this is the first exploration of robust IQA model learning from noisy low-cost labels. Theoretical analysis and extensive experiments on four popular IQA datasets show that the proposed method is robust toward different bias rates and annotation numbers and significantly outperforms the other learning based IQA models when only LC-MOS is available. Furthermore, we also achieve comparable performance with respect to the other models learned with LA-MOS. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Cross-modal Cognitive Consensus guided Audio-Visual Segmentation

Authors: Zhaofeng Shi, Qingbo Wu, Fanman Meng, Linfeng Xu, Hongliang Li

Abstract: Audio-Visual Segmentation (AVS) aims to extract the sounding object from a video frame, which is represented by a pixel-wise segmentation mask for application scenarios such as multi-modal video editing, augmented reality, and intelligent robot systems. The pioneering work conducts this task through dense feature-level audio-visual interaction, which ignores the dimension gap between different mod… ▽ More Audio-Visual Segmentation (AVS) aims to extract the sounding object from a video frame, which is represented by a pixel-wise segmentation mask for application scenarios such as multi-modal video editing, augmented reality, and intelligent robot systems. The pioneering work conducts this task through dense feature-level audio-visual interaction, which ignores the dimension gap between different modalities. More specifically, the audio clip could only provide a Global semantic label in each sequence, but the video frame covers multiple semantic objects across different Local regions, which leads to mislocalization of the representationally similar but semantically different object. In this paper, we propose a Cross-modal Cognitive Consensus guided Network (C3N) to align the audio-visual semantics from the global dimension and progressively inject them into the local regions via an attention mechanism. Firstly, a Cross-modal Cognitive Consensus Inference Module (C3IM) is developed to extract a unified-modal label by integrating audio/visual classification confidence and similarities of modality-agnostic label embeddings. Then, we feed the unified-modal label back to the visual backbone as the explicit semantic-level guidance via a Cognitive Consensus guided Attention Module (CCAM), which highlights the local features corresponding to the interested object. Extensive experiments on the Single Sound Source Segmentation (S4) setting and Multiple Sound Source Segmentation (MS3) setting of the AVSBench dataset demonstrate the effectiveness of the proposed method, which achieves state-of-the-art performance. Code will be available at https://github.com/ZhaofengSHI/AVS-C3N once accepted. △ Less

Submitted 8 May, 2024; v1 submitted 9 October, 2023; originally announced October 2023.

Comments: 14 pages

MSC Class: 68U10 ACM Class: I.4.6

Displacement field calculation of large-scale structures using computer vision with physical constraints

Authors: Yapeng Guo, Peng Zhong, Yi Zhuo, Fanzeng Meng, Hao Di, Shunlong Li

Abstract: Because of the advantages of easy deployment, low cost and non-contact, computer vision-based structural displacement acquisition technique has received wide attention and research in recent years. However, the displacement field acquisition of large-scale structures is a challenging topic due to the contradiction of camera field of view and resolution. This paper presents a large-scale structural… ▽ More Because of the advantages of easy deployment, low cost and non-contact, computer vision-based structural displacement acquisition technique has received wide attention and research in recent years. However, the displacement field acquisition of large-scale structures is a challenging topic due to the contradiction of camera field of view and resolution. This paper presents a large-scale structural displacement field calculation framework with integrated computer vision and physical constraints using only one camera. Firstly, the full-field image of the large-scale structure is obtained by processing the multi-view image using image stitching technique; secondly, the full-field image is meshed and the node displacements are calculated using an improved template matching method; and finally, the non-node displacements are described using shape functions considering physical constraints. The developed framework was validated using a scaled bridge model and evaluated by the proposed evaluation index for displacement field calculation accuracy. This paper can provide an effective way to obtain displacement fields of large-scale structures efficiently and cost-effectively. △ Less

Submitted 31 March, 2023; v1 submitted 28 March, 2023; originally announced March 2023.

Learning-Aided Beam Prediction in mmWave MU-MIMO Systems for High-Speed Railway

Authors: Fan Meng, Shengheng Liu, Yongming Huang, Zhaohua Lu

Abstract: The problem of beam alignment and tracking in high mobility scenarios such as high-speed railway (HSR) becomes extremely challenging, since large overhead cost and significant time delay are introduced for fast time-varying channel estimation. To tackle this challenge, we propose a learning-aided beam prediction scheme for HSR networks, which predicts the beam directions and the channel amplitudes… ▽ More The problem of beam alignment and tracking in high mobility scenarios such as high-speed railway (HSR) becomes extremely challenging, since large overhead cost and significant time delay are introduced for fast time-varying channel estimation. To tackle this challenge, we propose a learning-aided beam prediction scheme for HSR networks, which predicts the beam directions and the channel amplitudes within a period of future time with fine time granularity, using a group of observations. Concretely, we transform the problem of high-dimensional beam prediction into a two-stage task, i.e., a low-dimensional parameter estimation and a cascaded hybrid beamforming operation. In the first stage, the location and speed of a certain terminal are estimated by maximum likelihood criterion, and a data-driven data fusion module is designed to improve the final estimation accuracy and robustness. Then, the probable future beam directions and channel amplitudes are predicted, based on the HSR scenario priors including deterministic trajectory, motion model, and channel model. Furthermore, we incorporate a learnable non-linear map** module into the overall beam prediction to allow non-linear tracks. Both of the proposed learnable modules are model-based and have a good interpretability. Compared to the existing beam management scheme, the proposed beam prediction has (near) zero overhead cost and time delay. Simulation results verify the effectiveness of the proposed scheme. △ Less

Submitted 30 June, 2022; originally announced June 2022.

Comments: 14 pages, 10 figures

Journal ref: publised on IEEE Transactions on Communications, 70(1): 693-706, (2022)

Learnable Model-Driven Performance Prediction and Optimization for Imperfect MIMO System: Framework and Application

Authors: Fan Meng, Shengheng Liu, Yongming Huang, Zhaohua Lu

Abstract: State-of-the-art schemes for performance analysis and optimization of multiple-input multiple-output systems generally experience degradation or even become invalid in dynamic complex scenarios with unknown interference and channel state information (CSI) uncertainty. To adapt to the challenging settings and better accomplish these network auto-tuning tasks, we propose a generic learnable model-dr… ▽ More State-of-the-art schemes for performance analysis and optimization of multiple-input multiple-output systems generally experience degradation or even become invalid in dynamic complex scenarios with unknown interference and channel state information (CSI) uncertainty. To adapt to the challenging settings and better accomplish these network auto-tuning tasks, we propose a generic learnable model-driven framework in this paper. To explain how the proposed framework works, we consider regularized zero-forcing precoding as a usage instance and design a light-weight neural network for refined prediction of sum rate and detection error based on coarse model-driven approximations. Then, we estimate the CSI uncertainty on the learned predictor in an iterative manner and, on this basis, optimize the transmit regularization term and subsequent receive power scaling factors. A deep unfolded projected gradient descent based algorithm is proposed for power scaling, which achieves favorable trade-off between convergence rate and robustness. △ Less

Submitted 30 June, 2022; originally announced June 2022.

Comments: 30 pages, 9 figures, submitted to IEEE Transaction on Wireless Communications (major revision)

Universal Learned Image Compression With Low Computational Cost

Authors: Bowen Li, Yao Xin, Youneng Bao, Fanyang Meng, Yongsheng Liang, Wen Tan

Abstract: Recently, learned image compression methods have developed rapidly and exhibited excellent rate-distortion performance when compared to traditional standards, such as JPEG, JPEG2000 and BPG. However, the learning-based methods suffer from high computational costs, which is not beneficial for deployment on devices with limited resources. To this end, we propose shift-addition parallel modules (SAPM… ▽ More Recently, learned image compression methods have developed rapidly and exhibited excellent rate-distortion performance when compared to traditional standards, such as JPEG, JPEG2000 and BPG. However, the learning-based methods suffer from high computational costs, which is not beneficial for deployment on devices with limited resources. To this end, we propose shift-addition parallel modules (SAPMs), including SAPM-E for the encoder and SAPM-D for the decoder, to largely reduce the energy consumption. To be specific, they can be taken as plug-and-play components to upgrade existing CNN-based architectures, where the shift branch is used to extract large-grained features as compared to small-grained features learned by the addition branch. Furthermore, we thoroughly analyze the probability distribution of latent representations and propose to use Laplace Mixture Likelihoods for more accurate entropy estimation. Experimental results demonstrate that the proposed methods can achieve comparable or even better performance on both PSNR and MS-SSIM metrics to that of the convolutional counterpart with an about 2x energy reduction. △ Less

Submitted 23 June, 2022; originally announced June 2022.

Comments: 5 pages

Transformations in Learned Image Compression from a Modulation Perspective

Authors: Youneng Bao, Fangyang Meng, Wen Tan, Chao Li, Yonghong Tian, Yongsheng Liang

Abstract: In this paper, a unified transformation method in learned image compression(LIC) is proposed from the perspective of modulation. Firstly, the quantization in LIC is considered as a generalized channel with additive uniform noise. Moreover, the LIC is interpreted as a particular communication system according to the consistency in structures and optimization objectives. Thus, the technology of comm… ▽ More In this paper, a unified transformation method in learned image compression(LIC) is proposed from the perspective of modulation. Firstly, the quantization in LIC is considered as a generalized channel with additive uniform noise. Moreover, the LIC is interpreted as a particular communication system according to the consistency in structures and optimization objectives. Thus, the technology of communication systems can be applied to guide the design of modules in LIC. Furthermore, a unified transform method based on signal modulation (TSM) is defined. In the view of TSM, the existing transformation methods are mathematically reduced to a linear modulation. A series of transformation methods, e.g. TPM and TJM, are obtained by extending to nonlinear modulation. The experimental results on various datasets and backbone architectures verify that the effectiveness and robustness of the proposed method. More importantly, it further confirms the feasibility of guiding LIC design from a communication perspective. For example, when backbone architecture is hyperprior combining context model, our method achieves 3.52$\%$ BD-rate reduction over GDN on Kodak dataset without increasing complexity. △ Less

Submitted 12 March, 2024; v1 submitted 4 March, 2022; originally announced March 2022.

Comments: 10 pages, 8 figures

Improving Adversarial Waveform Generation based Singing Voice Conversion with Harmonic Signals

Authors: Haohan Guo, Zhi** Zhou, Fanbo Meng, Kai Liu

Abstract: Adversarial waveform generation has been a popular approach as the backend of singing voice conversion (SVC) to generate high-quality singing audio. However, the instability of GAN also leads to other problems, such as pitch jitters and U/V errors. It affects the smoothness and continuity of harmonics, hence degrades the conversion quality seriously. This paper proposes to feed harmonic signals to… ▽ More Adversarial waveform generation has been a popular approach as the backend of singing voice conversion (SVC) to generate high-quality singing audio. However, the instability of GAN also leads to other problems, such as pitch jitters and U/V errors. It affects the smoothness and continuity of harmonics, hence degrades the conversion quality seriously. This paper proposes to feed harmonic signals to the SVC model in advance to enhance audio generation. We extract the sine excitation from the pitch, and filter it with a linear time-varying (LTV) filter estimated by a neural network. Both these two harmonic signals are adopted as the inputs to generate the singing waveform. In our experiments, two mainstream models, MelGAN and ParallelWaveGAN, are investigated to validate the effectiveness of the proposed approach. We conduct a MOS test on clean and noisy test sets. The result shows that both signals significantly improve SVC in fidelity and timbre similarity. Besides, the case analysis further validates that this method enhances the smoothness and continuity of harmonics in the generated audio, and the filtered excitation better matches the target audio. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: Accepted by ICASSP 2022

FederatedNILM: A Distributed and Privacy-preserving Framework for Non-intrusive Load Monitoring based on Federated Deep Learning

Authors: Shuang Dai, Fanlin Meng, Qian Wang, Xizhong Chen

Abstract: Non-intrusive load monitoring (NILM), which usually utilizes machine learning methods and is effective in disaggregating smart meter readings from the household-level into appliance-level consumptions, can help to analyze electricity consumption behaviours of users and enable practical smart energy and smart grid applications. However, smart meters are privately owned and distributed, which make r… ▽ More Non-intrusive load monitoring (NILM), which usually utilizes machine learning methods and is effective in disaggregating smart meter readings from the household-level into appliance-level consumptions, can help to analyze electricity consumption behaviours of users and enable practical smart energy and smart grid applications. However, smart meters are privately owned and distributed, which make real-world applications of NILM challenging. To this end, this paper develops a distributed and privacy-preserving federated deep learning framework for NILM (FederatedNILM), which combines federated learning with a state-of-the-art deep learning architecture to conduct NILM for the classification of typical states of household appliances. Through extensive comparative experiments, the effectiveness of the proposed FederatedNILM framework is demonstrated. △ Less

Submitted 8 August, 2021; originally announced August 2021.

Electrical peak demand forecasting- A review

Authors: Shuang Dai, Fanlin Meng, Hongsheng Dai, Qian Wang, Xizhong Chen

Abstract: The power system is undergoing rapid evolution with the roll-out of advanced metering infrastructure and local energy applications (e.g. electric vehicles) as well as the increasing penetration of intermittent renewable energy at both transmission and distribution level, which characterizes the peak load demand with stronger randomness and less predictability and therefore poses a threat to the po… ▽ More The power system is undergoing rapid evolution with the roll-out of advanced metering infrastructure and local energy applications (e.g. electric vehicles) as well as the increasing penetration of intermittent renewable energy at both transmission and distribution level, which characterizes the peak load demand with stronger randomness and less predictability and therefore poses a threat to the power grid security. Since storing large quantities of electricity to satisfy load demand is neither economically nor environmentally friendly, effective peak demand management strategies and reliable peak load forecast methods become essential for optimizing the power system operations. To this end, this paper provides a timely and comprehensive overview of peak load demand forecast methods in the literature. To our best knowledge, this is the first comprehensive review on such topic. In this paper we first give a precise and unified problem definition of peak load demand forecast. Second, 139 papers on peak load forecast methods were systematically reviewed where methods were classified into different stages based on the timeline. Thirdly, a comparative analysis of peak load forecast methods are summarized and different optimizing methods to improve the forecast performance are discussed. The paper ends with a comprehensive summary of the reviewed papers and a discussion of potential future research directions. △ Less

Submitted 3 August, 2021; originally announced August 2021.

Multiple Dynamic Pricing for Demand Response with Adaptive Clustering-based Customer Segmentation in Smart Grids

Authors: Fanlin Meng, Qian Ma, Zixu Liu, Xiao-Jun Zeng

Abstract: In this paper, we propose a realistic multiple dynamic pricing approach to demand response in the retail market. First, an adaptive clustering-based customer segmentation framework is proposed to categorize customers into different groups to enable the effective identification of usage patterns. Second, customized demand models with important market constraints which capture the price-demand relat… ▽ More In this paper, we propose a realistic multiple dynamic pricing approach to demand response in the retail market. First, an adaptive clustering-based customer segmentation framework is proposed to categorize customers into different groups to enable the effective identification of usage patterns. Second, customized demand models with important market constraints which capture the price-demand relationship explicitly, are developed for each group of customers to improve the model accuracy and enable meaningful pricing. Third, the multiple pricing based demand response is formulated as a profit maximization problem subject to realistic market constraints. The overall aim of the proposed scalable and practical method aims to achieve 'right' prices for 'right' customers so as to benefit various stakeholders in the system such as grid operators, customers and retailers. The proposed multiple pricing framework is evaluated via simulations based on real-world datasets. △ Less

Submitted 10 June, 2021; originally announced June 2021.

Kalman-based interacting multiple-model wind speed estimator for wind turbines

Authors: Wai Hou Lio, Fanzhong Meng

Abstract: The use of state estimation technique offers a means of inferring the rotor-effective wind speed based upon solely standard measurements of the turbine. For the ease of design and computational concerns, such estimators are typically built based upon simplified turbine models that characterise the turbine with rigid blades. Large model mismatch, particularly in the power coefficient, could lead to… ▽ More The use of state estimation technique offers a means of inferring the rotor-effective wind speed based upon solely standard measurements of the turbine. For the ease of design and computational concerns, such estimators are typically built based upon simplified turbine models that characterise the turbine with rigid blades. Large model mismatch, particularly in the power coefficient, could lead to degradation in estimation performance. Therefore, in order to effectively reduce the adverse impact of parameter uncertainties in the estimator model, this paper develops a wind sped estimator based on the concept of interacting multiple-model adaptive estimation. The proposed estimator is composed of a bank of extended Kalman filters and each filter model is developed based on different power coefficient map** to match the operating turbine parameter. Subsequently, the algorithm combines the wind speed estimates provided by each filter based on their statistical properties. In addition, the proposed estimator not only can infer the rotor-effective wind speed, but also the uncertain system parameters, namely, the power coefficient. Simulation results demonstrate the proposed estimator achieved better improvement in estimating the rotor-effective wind speed and power coefficient compared to the standard Kalman filter approach. △ Less

Submitted 20 April, 2021; originally announced April 2021.

Comments: 6 pages, 7 figures, Accepted in IFAC World Congress 2020, in Berlin, Germany

CSLNSpeech: solving extended speech separation problem with the help of Chinese sign language

Authors: Jiasong Wu, Xuan Li, Taotao Li, Fanman Meng, Youyong Kong, Guanyu Yang, Lotfi Senhadji, Huazhong Shu

Abstract: Previous audio-visual speech separation methods use the synchronization of the speaker's facial movement and speech in the video to supervise the speech separation in a self-supervised way. In this paper, we propose a model to solve the speech separation problem assisted by both face and sign language, which we call the extended speech separation problem. We design a general deep learning network… ▽ More Previous audio-visual speech separation methods use the synchronization of the speaker's facial movement and speech in the video to supervise the speech separation in a self-supervised way. In this paper, we propose a model to solve the speech separation problem assisted by both face and sign language, which we call the extended speech separation problem. We design a general deep learning network for learning the combination of three modalities, audio, face, and sign language information, for better solving the speech separation problem. To train the model, we introduce a large-scale dataset named the Chinese Sign Language News Speech (CSLNSpeech) dataset, in which three modalities of audio, face, and sign language coexist. Experiment results show that the proposed model has better performance and robustness than the usual audio-visual system. Besides, sign language modality can also be used alone to supervise speech separation tasks, and the introduction of sign language is helpful for hearing-impaired people to learn and communicate. Last, our model is a general speech separation framework and can achieve very competitive separation performance on two open-source audio-visual datasets. The code is available at https://github.com/iveveive/SLNSpeech △ Less

Submitted 2 November, 2023; v1 submitted 21 July, 2020; originally announced July 2020.

Comments: 13 pages, 6 figures, 5 tables

Subjective and Objective De-raining Quality Assessment Towards Authentic Rain Image

Authors: Qingbo Wu, Lei Wang, King N. Ngan, Hongliang Li, Fanman Meng, Linfeng Xu

Abstract: Images acquired by outdoor vision systems easily suffer poor visibility and annoying interference due to the rainy weather, which brings great challenge for accurately understanding and describing the visual contents. Recent researches have devoted great efforts on the task of rain removal for improving the image visibility. However, there is very few exploration about the quality assessment of de… ▽ More Images acquired by outdoor vision systems easily suffer poor visibility and annoying interference due to the rainy weather, which brings great challenge for accurately understanding and describing the visual contents. Recent researches have devoted great efforts on the task of rain removal for improving the image visibility. However, there is very few exploration about the quality assessment of de-rained image, even it is crucial for accurately measuring the performance of various de-raining algorithms. In this paper, we first create a de-raining quality assessment (DQA) database that collects 206 authentic rain images and their de-rained versions produced by 6 representative single image rain removal algorithms. Then, a subjective study is conducted on our DQA database, which collects the subject-rated scores of all de-rained images. To quantitatively measure the quality of de-rained image with non-uniform artifacts, we propose a bi-directional feature embedding network (B-FEN) which integrates the features of global perception and local difference together. Experiments confirm that the proposed method significantly outperforms many existing universal blind image quality assessment models. To help the research towards perceptually preferred de-raining algorithm, we will publicly release our DQA database and B-FEN source code on https://github.com/wqb-uestc. △ Less

Submitted 5 October, 2019; v1 submitted 26 September, 2019; originally announced September 2019.

Comments: In this revision, we add the comparison with our previous exploration towards the de-raining quality assessment in Ref. [16]. Some typos in Tables III and IV are corrected, where the missed minus signs are added back for some OU metrics

An Integrated Optimization + Learning Approach to Optimal Dynamic Pricing for the Retailer with Multi-type Customers in Smart Grids

Authors: Fanlin Meng, Xiao-Jun Zeng, Yan Zhang, Chris J. Dent, Dunwei Gong

Abstract: In this paper, we consider a realistic and meaningful scenario in the context of smart grids where an electricity retailer serves three different types of customers, i.e., customers with an optimal home energy management system embedded in their smart meters (C-HEMS), customers with only smart meters (C-SM), and customers without smart meters (C-NONE). The main objective of this paper is to suppor… ▽ More In this paper, we consider a realistic and meaningful scenario in the context of smart grids where an electricity retailer serves three different types of customers, i.e., customers with an optimal home energy management system embedded in their smart meters (C-HEMS), customers with only smart meters (C-SM), and customers without smart meters (C-NONE). The main objective of this paper is to support the retailer to make optimal day-ahead dynamic pricing decisions in such a mixed customer pool. To this end, we propose a two-level decision-making framework where the retailer acting as upper-level agent firstly announces its electricity prices of next 24 hours and customers acting as lower-level agents subsequently schedule their energy usages accordingly. For the lower level problem, we model the price responsiveness of different customers according to their unique characteristics. For the upper level problem, we optimize the dynamic prices for the retailer to maximize its profit subject to realistic market constraints. The above two-level model is tackled by genetic algorithms (GA) based distributed optimization methods while its feasibility and effectiveness are confirmed via simulation results. △ Less

Submitted 21 March, 2018; v1 submitted 18 December, 2016; originally announced December 2016.

Comments: 38 pages, 6 figures

A Hybrid Optimization Approach to Demand Response Management for the Smart Grid

Authors: Fan-Lin Meng, Xiao-Jun Zeng

Abstract: This paper proposes a hybrid approach to optimal day-ahead pricing for demand response management. At the customer-side, compared with the existing work, a detailed, comprehensive and complete energy management system, which includes all possible types of appliances, all possible applications, and an effective waiting time cost model is proposed to manage the energy usages in households (lower lev… ▽ More This paper proposes a hybrid approach to optimal day-ahead pricing for demand response management. At the customer-side, compared with the existing work, a detailed, comprehensive and complete energy management system, which includes all possible types of appliances, all possible applications, and an effective waiting time cost model is proposed to manage the energy usages in households (lower level problem). At the retailer-side, the best retail prices are determined to maximize the retailer's profit (upper level problem). The interactions between the electricity retailer and its customers can be cast as a bilevel optimization problem. To overcome the weakness and infeasibility of conventional Karush--Kuhn--Tucker (KKT) approach for this particular type of bilevel problem, a hybrid pricing optimization approach, which adopts the multi-population genetic algorithms for the upper level problem and distributed individual optimization algorithms for the lower level problem, is proposed. Numerical results show the applicability and effectiveness of the proposed approach and its benefit to the retailer and its customers by improving the retailer's profit and reducing the customers' bills. △ Less

Submitted 28 October, 2015; originally announced October 2015.

A Survey of Quantum Lyapunov Control Methods

Authors: Shuang Cong, Fangfang Meng

Abstract: The condition of a quantum Lyapunov-based control which can be well used in a closed quantum system is that the method can make the system convergent but not just stable. In the convergence study of the quantum Lyapunov control, two situations are classified: non-degenerate cases and degenerate cases. In this paper, for these two situations, respectively, the target state is divided into four cate… ▽ More The condition of a quantum Lyapunov-based control which can be well used in a closed quantum system is that the method can make the system convergent but not just stable. In the convergence study of the quantum Lyapunov control, two situations are classified: non-degenerate cases and degenerate cases. In this paper, for these two situations, respectively, the target state is divided into four categories: eigenstate, the mixed state which commutes with the internal Hamiltonian, the superposition state, and the mixed state which does not commute with the internal Hamiltonian state. For these four categories, the quantum Lyapunov control methods for the closed quantum systems are summarized and analyzed. Especially, the convergence of the control system to the different target states is reviewed, and how to make the convergence conditions be satisfied is summarized and analyzed. △ Less

Submitted 25 June, 2013; v1 submitted 15 April, 2013; originally announced April 2013.

Comments: 14

Journal ref: The Scientific World Journal,Volume 2013, Article ID 967529

Quantum Lyapunov Control Based on the Average Value of an Imaginary Mechanical Quantity

Authors: Shuang Cong, Fangfang Meng, Sen Kuang

Abstract: The convergence of closed quantum systems in the degenerate cases to the desired target state by using the quantum Lyapunov control based on the average value of an imaginary mechanical quantity is studied. On the basis of the existing methods which can only ensure the single-control Hamiltonian systems converge toward a set, we design the control laws to make the multi-control Hamiltonian systems… ▽ More The convergence of closed quantum systems in the degenerate cases to the desired target state by using the quantum Lyapunov control based on the average value of an imaginary mechanical quantity is studied. On the basis of the existing methods which can only ensure the single-control Hamiltonian systems converge toward a set, we design the control laws to make the multi-control Hamiltonian systems converge to the desired target state. The convergence of the control system is proved, and the convergence to the desired target state is analyzed. How to make these conditions of convergence to the target state to be satisfied is proved or analyzed. Finally, numerical simulations for a three level system in the degenrate case transfering form an initial eigenstate to a target superposition state are studied to verify the effectiveness of the proposed control method. △ Less

Submitted 17 May, 2013; v1 submitted 5 February, 2013; originally announced February 2013.

Comments: 14 pages, 2 figures

Journal ref: Preprint of the 19th World Congress of the International Federation of Automation Control, Cape Town, South Africa, Aug., 2014, pp. 9991-9997

Implicit Lyapunov Control for the Quantum Liouville Equation

Authors: Shuang Cong, Fangfang Meng, Jianxiu Liu

Abstract: A quantum system whose internal Hamiltonian is not strongly regular or/and control Hamiltonians are not full connected, are thought to be in the degenerate cases. In this paper, convergence problems of the multi-control Hamiltonians closed quantum systems in the degenerate cases are solved by introducing implicit function perturbations and choosing an implicit Lyapunov function based on the averag… ▽ More A quantum system whose internal Hamiltonian is not strongly regular or/and control Hamiltonians are not full connected, are thought to be in the degenerate cases. In this paper, convergence problems of the multi-control Hamiltonians closed quantum systems in the degenerate cases are solved by introducing implicit function perturbations and choosing an implicit Lyapunov function based on the average value of an imaginary mechanical quantity. For the diagonal and non-diagonal tar-get states, respectively, control laws are designed. The convergence of the control system is proved, and an explicit design principle of the imaginary mechanical quantity is proposed. By using the proposed method, the multi-control Hamiltonians closed quantum systems in the degenerate cases can converge from any initial state to an arbitrary target state unitarily equivalent to the initial state. Finally, numerical simulations are studied to verify the effectiveness of the proposed control method. △ Less

Submitted 14 December, 2012; originally announced December 2012.

Comments: 8 pages, 2 figures

MSC Class: 65M12

Journal ref: Control Theory and Informatics,Vol. 4, No. 6, pp. 21- 32, 2014