-
HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models
Authors:
Li Pang,
Xiangyu Rui,
Long Cui,
Hongzhong Wang,
Deyu Meng,
Xiangyong Cao
Abstract:
Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations and plays a vital role in downstream tasks. Existing model-based methods have limitations in accurately modeling the complex image characteristics with handcraft priors, and deep learning-based methods suffer from poor generalization ability. To alleviate these issues, this paper proposes an unsupervis…
▽ More
Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations and plays a vital role in downstream tasks. Existing model-based methods have limitations in accurately modeling the complex image characteristics with handcraft priors, and deep learning-based methods suffer from poor generalization ability. To alleviate these issues, this paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff), which restores the clean HSIs from the product of two low-rank components, i.e., the reduced image and the coefficient matrix. Specifically, the reduced image, which has a low spectral dimension, lies in the image field and can be inferred from our improved diffusion model where a new guidance function with total variation (TV) prior is designed to ensure that the reduced image can be well sampled. The coefficient matrix can be effectively pre-estimated based on singular value decomposition (SVD) and rank-revealing QR (RRQR) factorization. Furthermore, a novel exponential noise schedule is proposed to accelerate the restoration process (about 5$\times$ acceleration for denoising) with little performance decrease. Extensive experimental results validate the superiority of our method in both performance and speed on a variety of HSI restoration tasks, including HSI denoising, noisy HSI super-resolution, and noisy HSI inpainting. The code is available at https://github.com/LiPang/HIRDiff.
△ Less
Submitted 24 February, 2024;
originally announced February 2024.
-
HIDFlowNet: A Flow-Based Deep Network for Hyperspectral Image Denoising
Authors:
Li Pang,
Weizhen Gu,
Xiangyong Cao,
Xiangyu Rui,
Jiangjun Peng,
Shuang Xu,
Gang Yang,
Deyu Meng
Abstract:
Hyperspectral image (HSI) denoising is essentially ill-posed since a noisy HSI can be degraded from multiple clean HSIs. However, current deep learning-based approaches ignore this fact and restore the clean image with deterministic map** (i.e., the network receives a noisy HSI and outputs a clean HSI). To alleviate this issue, this paper proposes a flow-based HSI denoising network (HIDFlowNet)…
▽ More
Hyperspectral image (HSI) denoising is essentially ill-posed since a noisy HSI can be degraded from multiple clean HSIs. However, current deep learning-based approaches ignore this fact and restore the clean image with deterministic map** (i.e., the network receives a noisy HSI and outputs a clean HSI). To alleviate this issue, this paper proposes a flow-based HSI denoising network (HIDFlowNet) to directly learn the conditional distribution of the clean HSI given the noisy HSI and thus diverse clean HSIs can be sampled from the conditional distribution. Overall, our HIDFlowNet is induced from the flow methodology and contains an invertible decoder and a conditional encoder, which can fully decouple the learning of low-frequency and high-frequency information of HSI. Specifically, the invertible decoder is built by staking a succession of invertible conditional blocks (ICBs) to capture the local high-frequency details since the invertible network is information-lossless. The conditional encoder utilizes down-sampling operations to obtain low-resolution images and uses transformers to capture correlations over a long distance so that global low-frequency information can be effectively extracted. Extensive experimental results on simulated and real HSI datasets verify the superiority of our proposed HIDFlowNet compared with other state-of-the-art methods both quantitatively and visually.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
Unsupervised Hyperspectral Pansharpening via Low-rank Diffusion Model
Authors:
Xiangyu Rui,
Xiangyong Cao,
Li Pang,
Zeyu Zhu,
Zongsheng Yue,
Deyu Meng
Abstract:
Hyperspectral pansharpening is a process of merging a high-resolution panchromatic (PAN) image and a low-resolution hyperspectral (LRHS) image to create a single high-resolution hyperspectral (HRHS) image. Existing Bayesian-based HS pansharpening methods require designing handcraft image prior to characterize the image features, and deep learning-based HS pansharpening methods usually require a la…
▽ More
Hyperspectral pansharpening is a process of merging a high-resolution panchromatic (PAN) image and a low-resolution hyperspectral (LRHS) image to create a single high-resolution hyperspectral (HRHS) image. Existing Bayesian-based HS pansharpening methods require designing handcraft image prior to characterize the image features, and deep learning-based HS pansharpening methods usually require a large number of paired training data and suffer from poor generalization ability. To address these issues, in this work, we propose a low-rank diffusion model for hyperspectral pansharpening by simultaneously leveraging the power of the pre-trained deep diffusion model and better generalization ability of Bayesian methods. Specifically, we assume that the HRHS image can be recovered from the product of two low-rank tensors, i.e., the base tensor and the coefficient matrix. The base tensor lies on the image field and has a low spectral dimension. Thus, we can conveniently utilize a pre-trained remote sensing diffusion model to capture its image structures. Additionally, we derive a simple yet quite effective way to pre-estimate the coefficient matrix from the observed LRHS image, which preserves the spectral information of the HRHS. Experimental results demonstrate that the proposed method performs better than some popular traditional approaches and gains better generalization ability than some DL-based methods. The code is released in https://github.com/xyrui/PLRDiff.
△ Less
Submitted 19 November, 2023; v1 submitted 18 May, 2023;
originally announced May 2023.
-
A Remote Baby Surveillance System with RFID and GPS Tracking
Authors:
Ruven A/L Sundarajoo,
Gwo Chin Chung,
Wai Leong Pang,
Soo Fun Tan
Abstract:
In the 21st century, sending babies or children to daycare centres has become more and more common among young guardians. The balance between full-time work and child care is increasingly challenging nowadays. In Malaysia, thousands of child abuse cases have been reported from babysitting centres every year, which indeed triggers the anxiety and stress of the guardians. Hence, this paper proposes…
▽ More
In the 21st century, sending babies or children to daycare centres has become more and more common among young guardians. The balance between full-time work and child care is increasingly challenging nowadays. In Malaysia, thousands of child abuse cases have been reported from babysitting centres every year, which indeed triggers the anxiety and stress of the guardians. Hence, this paper proposes to construct a remote baby surveillance system with radio-frequency identification (RFID) and global positioning system (GPS) tracking. With the incorporation of the Internet of Things (IoT), a sensor-based microcontroller is used to detect the conditions of the baby as well as the surrounding environment and then display the real-time data as well as notifications to alert the guardians via a mobile application. These conditions include the crying and waking of the baby, as well as temperature, the mattress's wetness, and moving objects around the baby. In addition, RFID and GPS location tracking are implemented to ensure the safety of the baby, while white noise is used to increase the comfort of the baby. In the end, a prototype has been successfully developed for functionality and reliability testing. Several experiments have been conducted to measure the efficiency of the mattress's wetness detection, the RFID transmission range, the frequency spectrum of white noise, and also the output power of the solar panel. The proposed system is expected to assist guardians in ensuring the safety and comfort of their babies remotely, as well as prevent any occurrence of child abuse.
△ Less
Submitted 26 November, 2022;
originally announced November 2022.
-
Audio-Visual Object Classification for Human-Robot Collaboration
Authors:
A. Xompero,
Y. L. Pang,
T. Patten,
A. Prabhakar,
B. Calli,
A. Cavallaro
Abstract:
Human-robot collaboration requires the contactless estimation of the physical properties of containers manipulated by a person, for example while pouring content in a cup or moving a food box. Acoustic and visual signals can be used to estimate the physical properties of such objects, which may vary substantially in shape, material and size, and also be occluded by the hands of the person. To faci…
▽ More
Human-robot collaboration requires the contactless estimation of the physical properties of containers manipulated by a person, for example while pouring content in a cup or moving a food box. Acoustic and visual signals can be used to estimate the physical properties of such objects, which may vary substantially in shape, material and size, and also be occluded by the hands of the person. To facilitate comparisons and stimulate progress in solving this problem, we present the CORSMAL challenge and a dataset to assess the performance of the algorithms through a set of well-defined performance scores. The tasks of the challenge are the estimation of the mass, capacity, and dimensions of the object (container), and the classification of the type and amount of its content. A novel feature of the challenge is our real-to-simulation framework for visualising and assessing the impact of estimation errors in human-to-robot handovers.
△ Less
Submitted 3 March, 2022;
originally announced March 2022.
-
High-Speed Light Focusing through Scattering Medium by Cooperatively Accelerated Genetic Algorithm
Authors:
Shu Guo,
Lin Pang
Abstract:
We develop an accelerated Genetic Algorithm (GA) system constructed by the cooperation of field-programmable gate array (FPGA) and optimized parameters of the GA. We found the enhanced decay of mutation rate makes convergence of the GA much faster, enabling the parameter-induced acceleration of the GA. Furthermore, the accelerated configuration of the GA is programmed in FPGA to boost processing s…
▽ More
We develop an accelerated Genetic Algorithm (GA) system constructed by the cooperation of field-programmable gate array (FPGA) and optimized parameters of the GA. We found the enhanced decay of mutation rate makes convergence of the GA much faster, enabling the parameter-induced acceleration of the GA. Furthermore, the accelerated configuration of the GA is programmed in FPGA to boost processing speed at the hardware level without external computation devices. This system has ability to focus light through scattering medium within 4 seconds with robust noise resistance and stable repetition performance, which could be further reduced to millisecond level with advanced board configuration. This study solves the long-term limitation of the GA, it promotes the applications of the GA in dynamic scattering mediums, with the capability to tackle wavefront sha** in biological material.
△ Less
Submitted 29 November, 2021;
originally announced November 2021.
-
Intelligent Scheduling and Power Control for Multimedia Transmission in 5G CoMP Systems: A Dynamic Bargaining Game
Authors:
Liying Li,
Chungang Yang,
Mbazingwa E. Mkiramweni,
Lei Pang
Abstract:
Intelligent terminals support a large number of multimedia, such as picture, audio, video, and so on. The coexistence of various multimedia makes it necessary to provide service for different requests. In this work, we consider interference-aware coordinated multi-point (CoMP) to mitigate inter-cell interference and improve total throughput in the fifth-generation (5G) mobile networks. To select t…
▽ More
Intelligent terminals support a large number of multimedia, such as picture, audio, video, and so on. The coexistence of various multimedia makes it necessary to provide service for different requests. In this work, we consider interference-aware coordinated multi-point (CoMP) to mitigate inter-cell interference and improve total throughput in the fifth-generation (5G) mobile networks. To select the scheduled edge users, cluster the cooperative base stations (BSs), and determine the transmitting power, a novel dynamic bargaining approach is proposed. Based on affinity propagation, we first select the users to be scheduled and the cooperative BSs serving them respectively. Then, based on the Nash bargaining solution (NBS), we develop a power control scheme considering the transmission delay, which guarantees a generalized proportional fairness among users. Simulation results demonstrate the superiority of the user-centric scheduling and power control methods in 5G CoMP systems.
△ Less
Submitted 9 May, 2019;
originally announced May 2019.
