Showing 1–29 of 29 results for author: Zeng, T

Multiple Latent Space Map** for Compressed Dark Image Enhancement

Authors: Yi Zeng, Zhengning Wang, Yuxuan Liu, Tianjiao Zeng, Xuhang Liu, Xinglong Luo, Shuaicheng Liu, Shuyuan Zhu, Bing Zeng

Abstract: Dark image enhancement aims at converting dark images to normal-light images. Existing dark image enhancement methods take uncompressed dark images as inputs and achieve great performance. However, in practice, dark images are often compressed before storage or transmission over the Internet. Current methods get poor performance when processing compressed dark images. Artifacts hidden in the dark… ▽ More Dark image enhancement aims at converting dark images to normal-light images. Existing dark image enhancement methods take uncompressed dark images as inputs and achieve great performance. However, in practice, dark images are often compressed before storage or transmission over the Internet. Current methods get poor performance when processing compressed dark images. Artifacts hidden in the dark regions are amplified by current methods, which results in uncomfortable visual effects for observers. Based on this observation, this study aims at enhancing compressed dark images while avoiding compression artifacts amplification. Since texture details intertwine with compression artifacts in compressed dark images, detail enhancement and blocking artifacts suppression contradict each other in image space. Therefore, we handle the task in latent space. To this end, we propose a novel latent map** network based on variational auto-encoder (VAE). Firstly, different from previous VAE-based methods with single-resolution features only, we exploit multiple latent spaces with multi-resolution features, to reduce the detail blur and improve image fidelity. Specifically, we train two multi-level VAEs to project compressed dark images and normal-light images into their latent spaces respectively. Secondly, we leverage a latent map** network to transform features from compressed dark space to normal-light space. Specifically, since the degradation models of darkness and compression are different from each other, the latent map** process is divided map** into enlightening branch and deblocking branch. Comprehensive experiments demonstrate that the proposed method achieves state-of-the-art performance in compressed dark image enhancement. △ Less

Submitted 12 March, 2024; originally announced March 2024.

Dynamic Multimodal Information Bottleneck for Multimodality Classification

Authors: Yingying Fang, Shuang Wu, Sheng Zhang, Chaoyan Huang, Tieyong Zeng, Xiaodan Xing, Simon Walsh, Guang Yang

Abstract: Effectively leveraging multimodal data such as various images, laboratory tests and clinical information is gaining traction in a variety of AI-based medical diagnosis and prognosis tasks. Most existing multi-modal techniques only focus on enhancing their performance by leveraging the differences or shared features from various modalities and fusing feature across different modalities. These appro… ▽ More Effectively leveraging multimodal data such as various images, laboratory tests and clinical information is gaining traction in a variety of AI-based medical diagnosis and prognosis tasks. Most existing multi-modal techniques only focus on enhancing their performance by leveraging the differences or shared features from various modalities and fusing feature across different modalities. These approaches are generally not optimal for clinical settings, which pose the additional challenges of limited training data, as well as being rife with redundant data or noisy modality channels, leading to subpar performance. To address this gap, we study the robustness of existing methods to data redundancy and noise and propose a generalized dynamic multimodal information bottleneck framework for attaining a robust fused feature representation. Specifically, our information bottleneck module serves to filter out the task-irrelevant information and noises in the fused feature, and we further introduce a sufficiency loss to prevent drop** of task-relevant information, thus explicitly preserving the sufficiency of prediction information in the distilled feature. We validate our model on an in-house and a public COVID19 dataset for mortality prediction as well as two public biomedical datasets for diagnostic tasks. Extensive experiments show that our method surpasses the state-of-the-art and is significantly more robust, being the only method to remain performance when large-scale noisy channels exist. Our code is publicly available at https://github.com/ayanglab/DMIB. △ Less

Submitted 25 November, 2023; v1 submitted 2 November, 2023; originally announced November 2023.

Comments: WACV 2024

VDIP-TGV: Blind Image Deconvolution via Variational Deep Image Prior Empowered by Total Generalized Variation

Authors: Tingting Wu, Zhiyan Du, Zhi Li, Feng-Lei Fan, Tieyong Zeng

Abstract: Recovering clear images from blurry ones with an unknown blur kernel is a challenging problem. Deep image prior (DIP) proposes to use the deep network as a regularizer for a single image rather than as a supervised model, which achieves encouraging results in the nonblind deblurring problem. However, since the relationship between images and the network architectures is unclear, it is hard to find… ▽ More Recovering clear images from blurry ones with an unknown blur kernel is a challenging problem. Deep image prior (DIP) proposes to use the deep network as a regularizer for a single image rather than as a supervised model, which achieves encouraging results in the nonblind deblurring problem. However, since the relationship between images and the network architectures is unclear, it is hard to find a suitable architecture to provide sufficient constraints on the estimated blur kernels and clean images. Also, DIP uses the sparse maximum a posteriori (MAP), which is insufficient to enforce the selection of the recovery image. Recently, variational deep image prior (VDIP) was proposed to impose constraints on both blur kernels and recovery images and take the standard deviation of the image into account during the optimization process by the variational principle. However, we empirically find that VDIP struggles with processing image details and tends to generate suboptimal results when the blur kernel is large. Therefore, we combine total generalized variational (TGV) regularization with VDIP in this paper to overcome these shortcomings of VDIP. TGV is a flexible regularization that utilizes the characteristics of partial derivatives of varying orders to regularize images at different scales, reducing oil painting artifacts while maintaining sharp edges. The proposed VDIP-TGV effectively recovers image edges and details by supplementing extra gradient information through TGV. Additionally, this model is solved by the alternating direction method of multipliers (ADMM), which effectively combines traditional algorithms and deep learning methods. Experiments show that our proposed VDIP-TGV surpasses various state-of-the-art models quantitatively and qualitatively. △ Less

Submitted 10 November, 2023; v1 submitted 30 October, 2023; originally announced October 2023.

Comments: 13 pages, 5 figures

DSFNet: Dual-GCN and Location-fused Self-attention with Weighted Fast Normalized Fusion for Polyps Segmentation

Authors: Juntong Fan, Debesh Jha, Tieyong Zeng, Dayang Wang

Abstract: Polyps segmentation poses a significant challenge in medical imaging due to the flat surface of polyps and their texture similarity to surrounding tissues. This similarity gives rise to difficulties in establishing a clear boundary between polyps and the surrounding mucosa, leading to complications such as local overexposure and the presence of bright spot reflections in imaging. To counter this p… ▽ More Polyps segmentation poses a significant challenge in medical imaging due to the flat surface of polyps and their texture similarity to surrounding tissues. This similarity gives rise to difficulties in establishing a clear boundary between polyps and the surrounding mucosa, leading to complications such as local overexposure and the presence of bright spot reflections in imaging. To counter this problem, we propose a new dual graph convolution network (Dual-GCN) and location self-attention mechanisms with weighted fast normalization fusion model, named DSFNet. First, we introduce a feature enhancement block module based on Dual-GCN module to enhance local spatial and structural information extraction with fine granularity. Second, we introduce a location fused self-attention module to enhance the model's awareness and capacity to capture global information. Finally, the weighted fast normalized fusion method with trainable weights is introduced to efficiently integrate the feature maps from encoder, bottleneck, and decoder, thus promoting information transmission and facilitating the semantic consistency. Experimental results show that the proposed model surpasses other state-of-the-art models in gold standard indicators, such as Dice, MAE, and IoU. Both quantitative and qualitative analysis indicate that the proposed model demonstrates exceptional capability in polyps segmentation and has great potential clinical significance. We have shared our code on anonymous website for evaluation. △ Less

Submitted 27 November, 2023; v1 submitted 15 August, 2023; originally announced August 2023.

Comments: 10 pages, 6 figures, 3 tables

Fast MRI Reconstruction via Edge Attention

Authors: Hanhui Yang, Juncheng Li, Lok Ming Lui, Shihui Ying, Jun Shi, Tieyong Zeng

Abstract: Fast and accurate MRI reconstruction is a key concern in modern clinical practice. Recently, numerous Deep-Learning methods have been proposed for MRI reconstruction, however, they usually fail to reconstruct sharp details from the subsampled k-space data. To solve this problem, we propose a lightweight and accurate Edge Attention MRI Reconstruction Network (EAMRI) to reconstruct images with edge… ▽ More Fast and accurate MRI reconstruction is a key concern in modern clinical practice. Recently, numerous Deep-Learning methods have been proposed for MRI reconstruction, however, they usually fail to reconstruct sharp details from the subsampled k-space data. To solve this problem, we propose a lightweight and accurate Edge Attention MRI Reconstruction Network (EAMRI) to reconstruct images with edge guidance. Specifically, we design an efficient Edge Prediction Network to directly predict accurate edges from the blurred image. Meanwhile, we propose a novel Edge Attention Module (EAM) to guide the image reconstruction utilizing the extracted edge priors, as inspired by the popular self-attention mechanism. EAM first projects the input image and edges into Q_image, K_edge, and V_image, respectively. Then EAM pairs the Q_image with K_edge along the channel dimension, such that 1) it can search globally for the high-frequency image features that are activated by the edge priors; 2) the overall computation burdens are largely reduced compared with the traditional spatial-wise attention. With the help of EAM, the predicted edge priors can effectively guide the model to reconstruct high-quality MR images with accurate edges. Extensive experiments show that our proposed EAMRI outperforms other methods with fewer parameters and can recover more accurate edges. △ Less

Submitted 22 April, 2023; originally announced April 2023.

Comments: 10 figures, 5 tables

Hierarchical Perception Adversarial Learning Framework for Compressed Sensing MRI

Authors: Zhifan Gao, Yifeng Guo, Jia**g Zhang, Tieyong Zeng, Guang Yang

Abstract: The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer… ▽ More The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this challenge, we propose a hierarchical perception adversarial learning framework (HP-ALF). HP-ALF can perceive the image information in the hierarchical mechanism: image-level perception and patch-level perception. The former can reduce the visual perception difference in the entire image, and thus achieve aliasing artifact removal. The latter can reduce this difference in the regions of the image, and thus recover fine details. Specifically, HP-ALF achieves the hierarchical mechanism by utilizing multilevel perspective discrimination. This discrimination can provide the information from two perspectives (overall and regional) for adversarial learning. It also utilizes a global and local coherent discriminator to provide structure information to the generator during training. In addition, HP-ALF contains a context-aware learning block to effectively exploit the slice information between individual images for better reconstruction performance. The experiments validated on three datasets demonstrate the effectiveness of HP-ALF and its superiority to the comparative methods. △ Less

Submitted 27 January, 2023; originally announced February 2023.

Comments: 15 pages, 13 figures, IEEE TMI

Robo-Chargers: Optimal Operation and Planning of a Robotic Charging System to Alleviate Overstay

Authors: Yi Ju, Teng Zeng, Zaid Allybokus, Scott Moura

Abstract: Charging infrastructure availability is a major concern for plug-in electric vehicle users. Nowadays, the limited public chargers are commonly occupied by vehicles which have already been fully charged. Such phenomenon, known as overstay, hinders other vehicles' accessibility to charging resources. In this paper, we analyze a charging facility innovation to tackle the challenge of overstay, levera… ▽ More Charging infrastructure availability is a major concern for plug-in electric vehicle users. Nowadays, the limited public chargers are commonly occupied by vehicles which have already been fully charged. Such phenomenon, known as overstay, hinders other vehicles' accessibility to charging resources. In this paper, we analyze a charging facility innovation to tackle the challenge of overstay, leveraging the idea of Robo-chargers - automated chargers that can rotate in a charging station and proactively plug or unplug plug-in electric vehicles. We formalize an operation model for stations incorporating Fixed-chargers and Robo-chargers. Optimal scheduling can be solved with the recognition of the combinatorial nature of vehicle-charger assignments, charging dynamics, and customer waiting behaviors. Then, with operation model nested, we develop a planning model to guide economical investment on both types of chargers so that the total cost of ownership is minimized. In the planning phase, it further considers charging demand variances and service capacity requirements. In this paper, we provide systematic techno-economical methods to evaluate if introducing Robo-chargers is beneficial given a specific application scenario. Comprehensive sensitivity analysis based on real-world data highlights the advantages of Robo-chargers, especially in a scenario where overstay is severe. Validations also suggest the tractability of operation model and robustness of planning results for real-time application under reasonable model mismatches, uncertainties and disturbances. △ Less

Submitted 18 June, 2023; v1 submitted 6 December, 2022; originally announced December 2022.

Journal ref: IEEE Transactions on Smart Grid

Shadow-Oriented Tracking Method for Multi-Target Tracking in Video-SAR

Authors: Xiaochuan Ni, Xiaoling Zhang, Xu Zhan, Zhenyu Yang, Jun Shi, Shunjun Wei, Tianjiao Zeng

Abstract: This work focuses on multi-target tracking in Video synthetic aperture radar. Specifically, we refer to tracking based on targets' shadows. Current methods have limited accuracy as they fail to consider shadows' characteristics and surroundings fully. Shades are low-scattering and varied, resulting in missed tracking. Surroundings can cause interferences, resulting in false tracking. To solve thes… ▽ More This work focuses on multi-target tracking in Video synthetic aperture radar. Specifically, we refer to tracking based on targets' shadows. Current methods have limited accuracy as they fail to consider shadows' characteristics and surroundings fully. Shades are low-scattering and varied, resulting in missed tracking. Surroundings can cause interferences, resulting in false tracking. To solve these, we propose a shadow-oriented multi-target tracking method (SOTrack). To avoid false tracking, a pre-processing module is proposed to enhance shadows from surroundings, thus reducing their interferences. To avoid missed tracking, a detection method based on deep learning is designed to thoroughly learn shadows' features, thus increasing the accurate estimation. And further, a recall module is designed to recall missed shadows. We conduct experiments on measured data. Results demonstrate that, compared with other methods, SOTrack achieves much higher performance in tracking accuracy-18.4%. And ablation study confirms the effectiveness of the proposed modules. △ Less

Submitted 29 November, 2022; originally announced November 2022.

A Model-data-driven Network Embedding Multidimensional Features for Tomographic SAR Imaging

Authors: Yu Ren, Xiaoling Zhang, Xu Zhan, Jun Shi, Shunjun Wei, Tianjiao Zeng

Abstract: Deep learning (DL)-based tomographic SAR imaging algorithms are gradually being studied. Typically, they use an unfolding network to mimic the iterative calculation of the classical compressive sensing (CS)-based methods and process each range-azimuth unit individually. However, only one-dimensional features are effectively utilized in this way. The correlation between adjacent resolution units is… ▽ More Deep learning (DL)-based tomographic SAR imaging algorithms are gradually being studied. Typically, they use an unfolding network to mimic the iterative calculation of the classical compressive sensing (CS)-based methods and process each range-azimuth unit individually. However, only one-dimensional features are effectively utilized in this way. The correlation between adjacent resolution units is ignored directly. To address that, we propose a new model-data-driven network to achieve tomoSAR imaging based on multi-dimensional features. Guided by the deep unfolding methodology, a two-dimensional deep unfolding imaging network is constructed. On the basis of it, we add two 2D processing modules, both convolutional encoder-decoder structures, to enhance multi-dimensional features of the imaging scene effectively. Meanwhile, to train the proposed multifeature-based imaging network, we construct a tomoSAR simulation dataset consisting entirely of simulation data of buildings. Experiments verify the effectiveness of the model. Compared with the conventional CS-based FISTA method and DL-based gamma-Net method, the result of our proposed method has better performance on completeness while having decent imaging accuracy. △ Less

Submitted 27 November, 2022; originally announced November 2022.

Near-filed SAR Image Restoration with Deep Learning Inverse Technique: A Preliminary Study

Authors: Xu Zhan, Xiaoling Zhang, Wensi Zhang, Jun Shi, Shunjun Wei, Tianjiao Zeng

Abstract: Benefiting from a relatively larger aperture's angle, and in combination with a wide transmitting bandwidth, near-field synthetic aperture radar (SAR) provides a high-resolution image of a target's scattering distribution-hot spots. Meanwhile, imaging result suffers inevitable degradation from sidelobes, clutters, and noises, hindering the information retrieval of the target. To restore the image,… ▽ More Benefiting from a relatively larger aperture's angle, and in combination with a wide transmitting bandwidth, near-field synthetic aperture radar (SAR) provides a high-resolution image of a target's scattering distribution-hot spots. Meanwhile, imaging result suffers inevitable degradation from sidelobes, clutters, and noises, hindering the information retrieval of the target. To restore the image, current methods make simplified assumptions; for example, the point spread function (PSF) is spatially consistent, the target consists of sparse point scatters, etc. Thus, they achieve limited restoration performance in terms of the target's shape, especially for complex targets. To address these issues, a preliminary study is conducted on restoration with the recent promising deep learning inverse technique in this work. We reformulate the degradation model into a spatially variable complex-convolution model, where the near-field SAR's system response is considered. Adhering to it, a model-based deep learning network is designed to restore the image. A simulated degraded image dataset from multiple complex target models is constructed to validate the network. All the images are formulated using the electromagnetic simulation tool. Experiments on the dataset reveal their effectiveness. Compared with current methods, superior performance is achieved regarding the target's shape and energy estimation. △ Less

Submitted 27 November, 2022; originally announced November 2022.

Solving 3D Radar Imaging Inverse Problems with a Multi-cognition Task-oriented Framework

Authors: Xu Zhan, Xiaoling Zhang, Mou Wang, Jun Shi, Shunjun Wei, Tianjiao Zeng

Abstract: This work focuses on 3D Radar imaging inverse problems. Current methods obtain undifferentiated results that suffer task-depended information retrieval loss and thus don't meet the task's specific demands well. For example, biased scattering energy may be acceptable for screen imaging but not for scattering diagnosis. To address this issue, we propose a new task-oriented imaging framework. The ima… ▽ More This work focuses on 3D Radar imaging inverse problems. Current methods obtain undifferentiated results that suffer task-depended information retrieval loss and thus don't meet the task's specific demands well. For example, biased scattering energy may be acceptable for screen imaging but not for scattering diagnosis. To address this issue, we propose a new task-oriented imaging framework. The imaging principle is task-oriented through an analysis phase to obtain task's demands. The imaging model is multi-cognition regularized to embed and fulfill demands. The imaging method is designed to be general-ized, where couplings between cognitions are decoupled and solved individually with approximation and variable-splitting techniques. Tasks include scattering diagnosis, person screen imaging, and parcel screening imaging are given as examples. Experiments on data from two systems indicate that the pro-posed framework outperforms the current ones in task-depended information retrieval. △ Less

Submitted 27 November, 2022; originally announced November 2022.

Retinex Image Enhancement Based on Sequential Decomposition With a Plug-and-Play Framework

Authors: Tingting Wu, Wenna Wu, Ying Yang, Feng-Lei Fan, Tieyong Zeng

Abstract: The Retinex model is one of the most representative and effective methods for low-light image enhancement. However, the Retinex model does not explicitly tackle the noise problem, and shows unsatisfactory enhancing results. In recent years, due to the excellent performance, deep learning models have been widely used in low-light image enhancement. However, these methods have two limitations: i) Th… ▽ More The Retinex model is one of the most representative and effective methods for low-light image enhancement. However, the Retinex model does not explicitly tackle the noise problem, and shows unsatisfactory enhancing results. In recent years, due to the excellent performance, deep learning models have been widely used in low-light image enhancement. However, these methods have two limitations: i) The desirable performance can only be achieved by deep learning when a large number of labeled data are available. However, it is not easy to curate massive low/normal-light paired data; ii) Deep learning is notoriously a black-box model [1]. It is difficult to explain their inner-working mechanism and understand their behaviors. In this paper, using a sequential Retinex decomposition strategy, we design a plug-and-play framework based on the Retinex theory for simultaneously image enhancement and noise removal. Meanwhile, we develop a convolutional neural network-based (CNN-based) denoiser into our proposed plug-and-play framework to generate a reflectance component. The final enhanced image is produced by integrating the illumination and reflectance with gamma correction. The proposed plug-and-play framework can facilitate both post hoc and ad hoc interpretability. Extensive experiments on different datasets demonstrate that our framework outcompetes the state-of-the-art methods in both image enhancement and denoising. △ Less

Submitted 17 February, 2023; v1 submitted 11 October, 2022; originally announced October 2022.

Spherical Image Inpainting with Frame Transformation and Data-driven Prior Deep Networks

Authors: Jianfei Li, Chaoyan Huang, Raymond Chan, Han Feng, Micheal Ng, Tieyong Zeng

Abstract: Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modelling, and medical imaging. It is non-trivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with deep learning-based regularizer. Instead of a naive… ▽ More Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modelling, and medical imaging. It is non-trivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regularizer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using deep learning denoiser and plug-and-play model. △ Less

Submitted 29 September, 2022; originally announced September 2022.

MSC Class: 68Q25; 68R10; 68U05

A Sub-pixel Accurate Quantification of Joint Space Narrowing Progression in Rheumatoid Arthritis

Authors: Yafei Ou, Prasoon Ambalathankandy, Ryunosuke Furuya, Seiya Kawada, Tianyu Zeng, Yujie An, Tamotsu Kamishima, Kenichi Tamura, Masayuki Ikebe

Abstract: Rheumatoid arthritis (RA) is a chronic autoimmune disease that primarily affects peripheral synovial joints, like fingers, wrist and feet. Radiology plays a critical role in the diagnosis and monitoring of RA. Limited by the current spatial resolution of radiographic imaging, joint space narrowing (JSN) progression of RA with the same reason above can be less than one pixel per year with universal… ▽ More Rheumatoid arthritis (RA) is a chronic autoimmune disease that primarily affects peripheral synovial joints, like fingers, wrist and feet. Radiology plays a critical role in the diagnosis and monitoring of RA. Limited by the current spatial resolution of radiographic imaging, joint space narrowing (JSN) progression of RA with the same reason above can be less than one pixel per year with universal spatial resolution. Insensitive monitoring of JSN can hinder the radiologist/rheumatologist from making a proper and timely clinical judgment. In this paper, we propose a novel and sensitive method that we call partial image phase-only correlation which aims to automatically quantify JSN progression in the early stages of RA. The majority of the current literature utilizes the mean error, root-mean-square deviation and standard deviation to report the accuracy at pixel level. Our work measures JSN progression between a baseline and its follow-up finger joint images by using the phase spectrum in the frequency domain. Using this study, the mean error can be reduced to 0.0130mm when applied to phantom radiographs with ground truth, and 0.0519mm standard deviation for clinical radiography. With its sub-pixel accuracy far beyond manual measurement, we are optimistic that our work is promising for automatically quantifying JSN progression. △ Less

Submitted 1 November, 2022; v1 submitted 19 May, 2022; originally announced May 2022.

Multiple Degradation and Reconstruction Network for Single Image Denoising via Knowledge Distillation

Authors: Juncheng Li, Hanhui Yang, Qiaosi Yi, Faming Fang, Guangwei Gao, Tieyong Zeng, Guixu Zhang

Abstract: Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multip… ▽ More Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer and more accurate features from heterogeneous models, which make it possible to reconstruct higher-quality denoised images under extreme conditions. Extensive experiments show that our MDRN achieves favorable performance against other SID models with fewer parameters. Meanwhile, plenty of ablation studies demonstrate that the introduced HMDS can improve the performance of tiny models or the model under high noise levels, which is extremely useful for related applications. △ Less

Submitted 29 April, 2022; originally announced April 2022.

Comments: Accepted by CVPR Workshop 2022

Wireless-Enabled Asynchronous Federated Fourier Neural Network for Turbulence Prediction in Urban Air Mobility (UAM)

Authors: Tengchan Zeng, Omid Semiari, Walid Saad, Mehdi Bennis

Abstract: To meet the growing mobility needs in intra-city transportation, the concept of urban air mobility (UAM) has been proposed in which vertical takeoff and landing (VTOL) aircraft are used to provide a ride-hailing service. In UAM, aircraft can operate in designated air spaces known as corridors, that link the aerodromes. A reliable communication network between GBSs and aircraft enables UAM to adequ… ▽ More To meet the growing mobility needs in intra-city transportation, the concept of urban air mobility (UAM) has been proposed in which vertical takeoff and landing (VTOL) aircraft are used to provide a ride-hailing service. In UAM, aircraft can operate in designated air spaces known as corridors, that link the aerodromes. A reliable communication network between GBSs and aircraft enables UAM to adequately utilize the airspace and create a fast, efficient, and safe transportation system. In this paper, to characterize the wireless connectivity performance for UAM, a spatial model is proposed. For this setup, the distribution of the distance between an arbitrarily selected GBS and its associated aircraft and the Laplace transform of the interference experienced by the GBS are derived. Using these results, the signal-to-interference ratio (SIR)-based connectivity probability is determined to capture the connectivity performance of the UAM aircraft-to-ground communication network. Then, leveraging these connectivity results, a wireless-enabled asynchronous federated learning (AFL) framework that uses a Fourier neural network is proposed to tackle the challenging problem of turbulence prediction during UAM operations. For this AFL scheme, a staleness-aware global aggregation scheme is introduced to expedite the convergence to the optimal turbulence prediction model used by UAM aircraft. Simulation results validate the theoretical derivations for the UAM wireless connectivity. The results also demonstrate that the proposed AFL framework converges to the optimal turbulence prediction model faster than the synchronous federated learning baselines and a staleness-free AFL approach. Furthermore, the results characterize the performance of wireless connectivity and convergence of the aircraft's turbulence model under different parameter settings, offering useful UAM design guidelines. △ Less

Submitted 26 December, 2021; originally announced January 2022.

Comments: 30 pages, 10 figures

Baihe: SysML Framework for AI-driven Databases

Authors: Andreas Pfadler, Rong Zhu, Wei Chen, Botong Huang, Tian**g Zeng, Bolin Ding, **gren Zhou

Abstract: We present Baihe, a SysML Framework for AI-driven Databases. Using Baihe, an existing relational database system may be retrofitted to use learned components for query optimization or other common tasks, such as e.g. learned structure for indexing. To ensure the practicality and real world applicability of Baihe, its high level architecture is based on the following requirements: separation from t… ▽ More We present Baihe, a SysML Framework for AI-driven Databases. Using Baihe, an existing relational database system may be retrofitted to use learned components for query optimization or other common tasks, such as e.g. learned structure for indexing. To ensure the practicality and real world applicability of Baihe, its high level architecture is based on the following requirements: separation from the core system, minimal third party dependencies, Robustness, stability and fault tolerance, as well as stability and configurability. Based on the high level architecture, we then describe a concrete implementation of Baihe for PostgreSQL and present example use cases for learned query optimizers. To serve both practitioners, as well as researchers in the DB and AI4DB community Baihe for PostgreSQL will be released under open source license. △ Less

Submitted 29 December, 2021; originally announced December 2021.

A Systematic Survey of Deep Learning-based Single-Image Super-Resolution

Authors: Juncheng Li, Zehua Pei, Wenjie Li, Guangwei Gao, Longguang Wang, Yingqian Wang, Tieyong Zeng

Abstract: Single-image super-resolution (SISR) is an important task in image processing, which aims to enhance the resolution of imaging systems. Recently, SISR has made a huge leap and has achieved promising results with the help of deep learning (DL). In this survey, we give an overview of DL-based SISR methods and group them according to their design targets. Specifically, we first introduce the problem… ▽ More Single-image super-resolution (SISR) is an important task in image processing, which aims to enhance the resolution of imaging systems. Recently, SISR has made a huge leap and has achieved promising results with the help of deep learning (DL). In this survey, we give an overview of DL-based SISR methods and group them according to their design targets. Specifically, we first introduce the problem definition, research background, and the significance of SISR. Secondly, we introduce some related works, including benchmark datasets, upsampling methods, optimization objectives, and image quality assessment methods. Thirdly, we provide a detailed investigation of SISR and give some domain-specific applications of it. Fourthly, we present the reconstruction results of some classic SISR methods to intuitively know their performance. Finally, we discuss some issues that still exist in SISR and summarize some new trends and future directions. This is an exhaustive survey of SISR, which can help researchers better understand SISR and inspire more exciting research in this field. An investigation project for SISR is provided at https://github.com/CV-JunchengLi/SISR-Survey. △ Less

Submitted 12 April, 2024; v1 submitted 29 September, 2021; originally announced September 2021.

Comments: 40 pages, 12 figures

An adaptive MPC scheme for energy-efficient control of building HVAC systems

Authors: Tingting Zeng, Prabir Barooah

Abstract: An autonomous adaptive MPC architecture is presented for control of heating, ventilation and air condition (HVAC) systems to maintain indoor temperature while reducing energy use. Although equipment use and occupant changes with time, existing MPC methods are not capable of automatically relearning models and computing control decisions reliably for extended periods without intervention from a hum… ▽ More An autonomous adaptive MPC architecture is presented for control of heating, ventilation and air condition (HVAC) systems to maintain indoor temperature while reducing energy use. Although equipment use and occupant changes with time, existing MPC methods are not capable of automatically relearning models and computing control decisions reliably for extended periods without intervention from a human expert. We seek to address this weakness. Two major features are embedded in the proposed architecture to enable autonomy: (i) a system identification algorithm from our prior work that periodically re-learns building dynamics and unmeasured internal heat loads from data without requiring re-tuning by experts. The estimated model is guaranteed to be stable and has desirable physical properties irrespective of the data; (ii) an MPC planner with a convex approximation of the original nonconvex problem. The planner uses a descent and convergent method, with the underlying optimization problem being feasible and convex. A year long simulation with a realistic plant shows that both of the features of the proposed architecture - periodic model and disturbance update and convexification of the planning problem - are essential to get the performance improvement over a commonly used baseline controller. Without these features, though MPC can outperform the baseline controller in certain situations, the benefits may not be substantial enough to warrant the investment in MPC. △ Less

Submitted 7 February, 2021; originally announced February 2021.

Comments: 12 pages, 7 figures

Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles

Authors: Tengchan Zeng, Omid Semiari, Mingzhe Chen, Walid Saad, Mehdi Bennis

Abstract: A new federated learning (FL) framework enabled by large-scale wireless connectivity is proposed for designing the autonomous controller of connected and autonomous vehicles (CAVs). In this framework, the learning models used by the controllers are collaboratively trained among a group of CAVs. To capture the varying CAV participation in the FL training process and the diverse local data quality a… ▽ More A new federated learning (FL) framework enabled by large-scale wireless connectivity is proposed for designing the autonomous controller of connected and autonomous vehicles (CAVs). In this framework, the learning models used by the controllers are collaboratively trained among a group of CAVs. To capture the varying CAV participation in the FL training process and the diverse local data quality among CAVs, a novel dynamic federated proximal (DFP) algorithm is proposed that accounts for the mobility of CAVs, the wireless fading channels, as well as the unbalanced and nonindependent and identically distributed data across CAVs. A rigorous convergence analysis is performed for the proposed algorithm to identify how fast the CAVs converge to using the optimal autonomous controller. In particular, the impacts of varying CAV participation in the FL process and diverse CAV data quality on the convergence of the proposed DFP algorithm are explicitly analyzed. Leveraging this analysis, an incentive mechanism based on contract theory is designed to improve the FL convergence speed. Simulation results using real vehicular data traces show that the proposed DFP-based controller can accurately track the target CAV speed over time and under different traffic scenarios. Moreover, the results show that the proposed DFP algorithm has a much faster convergence compared to popular FL algorithms such as federated averaging (FedAvg) and federated proximal (FedProx). The results also validate the feasibility of the contract-theoretic incentive mechanism and show that the proposed mechanism can improve the convergence speed of the DFP algorithm by 40% compared to the baselines. △ Less

Submitted 15 June, 2022; v1 submitted 5 February, 2021; originally announced February 2021.

Comments: 30 pages, 6 figures

Edge Adaptive Hybrid Regularization Model For Image Deblurring

Authors: Tingting Zhang, Jie Chen, Caiying Wu, Zhifei He, Tieyong Zeng, Qiyu **

Abstract: The parameter selection is crucial to regularization based image restoration methods. Generally speaking, a spatially fixed parameter for regularization item in the whole image does not perform well for both edge and smooth areas. A larger parameter of regularization item reduces noise better in smooth areas but blurs edge regions, while a small parameter sharpens edge but causes residual noise. I… ▽ More The parameter selection is crucial to regularization based image restoration methods. Generally speaking, a spatially fixed parameter for regularization item in the whole image does not perform well for both edge and smooth areas. A larger parameter of regularization item reduces noise better in smooth areas but blurs edge regions, while a small parameter sharpens edge but causes residual noise. In this paper, an automated spatially adaptive regularization model, which combines the harmonic and TV models, is proposed for reconstruction of noisy and blurred images. In the proposed model, it detects the edges and then spatially adjusts the parameters of Tikhonov and TV regularization terms for each pixel according to the edge information. Accordingly, the edge information matrix will be also dynamically updated during the iterations. Computationally, the newly-established model is convex, which can be solved by the semi-proximal alternating direction method of multipliers (sPADMM) with a linear-rate convergence rate. Numerical simulation results demonstrate that the proposed model effectively reserves the image edges and eliminates the noise and blur at the same time. In comparison to state-of-the-art algorithms, it outperforms other methods in terms of PSNR, SSIM and visual quality. △ Less

Submitted 6 April, 2021; v1 submitted 20 November, 2020; originally announced November 2020.

StereoNeuroBayesSLAM: A Neurobiologically Inspired Stereo Visual SLAM System Based on Direct Sparse Method

Authors: Tai** Zeng, Xiaoli Li, Bailu Si

Abstract: We propose a neurobiologically inspired visual simultaneous localization and map** (SLAM) system based on direction sparse method to real-time build cognitive maps of large-scale environments from a moving stereo camera. The core SLAM system mainly comprises a Bayesian attractor network, which utilizes neural responses of head direction (HD) cells in the hippocampus and grid cells in the medial… ▽ More We propose a neurobiologically inspired visual simultaneous localization and map** (SLAM) system based on direction sparse method to real-time build cognitive maps of large-scale environments from a moving stereo camera. The core SLAM system mainly comprises a Bayesian attractor network, which utilizes neural responses of head direction (HD) cells in the hippocampus and grid cells in the medial entorhinal cortex (MEC) to represent the head direction and the position of the robot in the environment, respectively. Direct sparse method is employed to accurately and robustly estimate velocity information from a stereo camera. Input rotational and translational velocities are integrated by the HD cell and grid cell networks, respectively. We demonstrated our neurobiologically inspired stereo visual SLAM system on the KITTI odometry benchmark datasets. Our proposed SLAM system is robust to real-time build a coherent semi-metric topological map from a stereo camera. Qualitative evaluation on cognitive maps shows that our proposed neurobiologically inspired stereo visual SLAM system outperforms our previous brain-inspired algorithms and the neurobiologically inspired monocular visual SLAM system both in terms of tracking accuracy and robustness, which is closer to the traditional state-of-the-art one. △ Less

Submitted 6 March, 2020; originally announced March 2020.

Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms

Authors: Tengchan Zeng, Omid Semiari, Mohammad Mozaffari, Mingzhe Chen, Walid Saad, Mehdi Bennis

Abstract: Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks ranging from coordinated trajectory planning to cooperative target recognition. However, due to the lack of continuous connections between the UAV swarm and ground base stations (BSs), using centralized ML will be challenging, particularly when dealing with a large volume of data. In this pape… ▽ More Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks ranging from coordinated trajectory planning to cooperative target recognition. However, due to the lack of continuous connections between the UAV swarm and ground base stations (BSs), using centralized ML will be challenging, particularly when dealing with a large volume of data. In this paper, a novel framework is proposed to implement distributed federated learning (FL) algorithms within a UAV swarm that consists of a leading UAV and several following UAVs. Each following UAV trains a local FL model based on its collected data and then sends this trained local model to the leading UAV who will aggregate the received models, generate a global FL model, and transmit it to followers over the intra-swarm network. To identify how wireless factors, like fading, transmission delay, and UAV antenna angle deviations resulting from wind and mechanical vibrations, impact the performance of FL, a rigorous convergence analysis for FL is performed. Then, a joint power allocation and scheduling design is proposed to optimize the convergence rate of FL while taking into account the energy consumption during convergence and the delay requirement imposed by the swarm's control system. Simulation results validate the effectiveness of the FL convergence analysis and show that the joint design strategy can reduce the number of communication rounds needed for convergence by as much as 35% compared with the baseline design. △ Less

Submitted 10 June, 2020; v1 submitted 19 February, 2020; originally announced February 2020.

Comments: 8 pages, 4 figures

Inducing Human Behavior to Alleviate Overstay at PEV Charging Station

Authors: Sangjae Bae, Teng Zeng, Bertrand Travacca, Scott Moura

Abstract: As the plug-in electric vehicle (PEV) market expands worldwide, PEV penetration has out-paced public PEV charging accessibility. In addition to charging infrastructure deployment, charging station operation is another key factor for improving charging service accessibility. In this paper, we propose a mathematical framework to optimally operate a PEV charging station, whose service capability is c… ▽ More As the plug-in electric vehicle (PEV) market expands worldwide, PEV penetration has out-paced public PEV charging accessibility. In addition to charging infrastructure deployment, charging station operation is another key factor for improving charging service accessibility. In this paper, we propose a mathematical framework to optimally operate a PEV charging station, whose service capability is constrained by the number of available chargers. This mathematical framework specifically exploits human behavioral modeling to alleviate the "overstaying" issue that occurs when a vehicle is fully charged. Our behavioral model effectively captures human decision-making when humans are exposed to multiple charging product options, which differ in both price and quality-of-service. We reformulate the associated non-convex problem to a multi-convex problem via the Young-Fenchel transform. We then apply the Block Coordinate Descent algorithm to efficiently solve the optimization problem. Numerical experiments illustrate the performance of the proposed method. Simulation results show that a station operator who leverages optimally priced charging options could realize benefits in three ways: (i) net profits gains, (ii) overstay reduction, and (iii) increased quality-of-service. △ Less

Submitted 4 December, 2019; originally announced December 2019.

Comments: Submitted to 2020 American Control Conference

Dependence Control for Reliability Optimization in Vehicular Networks

Authors: Tengchan Zeng, Omid Semiari, Walid Saad, Mehdi Bennis

Abstract: Vehicular networks will play an important role in enhancing road safety, improving transportation efficiency, and providing seamless Internet service for users on the road. Rea** the benefit of vehicular networks is contingent upon meeting stringent wireless communication performance requirements, particularly in terms of delay and reliability. In this paper, a dependence control mechanism is pr… ▽ More Vehicular networks will play an important role in enhancing road safety, improving transportation efficiency, and providing seamless Internet service for users on the road. Rea** the benefit of vehicular networks is contingent upon meeting stringent wireless communication performance requirements, particularly in terms of delay and reliability. In this paper, a dependence control mechanism is proposed to improve the overall reliability of vehicular networks. In particular, the dependence between the communication delays of different vehicle-to-vehicle (V2V) links is first modeled. Then, the concept of a concordance order, stemming from stochastic ordering theory, is introduced to show that a higher dependence can lead to a better reliability. Using this insight, a power allocation problem is formulated to maximize the concordance, thereby optimizing the overall communication reliability of the V2V system. To obtain an efficient solution to the power allocation problem, a dual update method is introduced. Simulation results verify the effectiveness of performing dependence control for reliability optimization in a vehicular network, and show that the proposed mechanism can achieve up to 25% reliability gain compared to a baseline system that uses a random power allocation. △ Less

Submitted 3 August, 2019; originally announced August 2019.

Total Variation and Tight Frame Image Segmentation with Intensity Inhomogeneity

Authors: Raymond Chan, Hongfei Yang, Tieyong Zeng

Abstract: Image segmentation is an important task in the domain of computer vision and medical imaging. In natural and medical images, intensity inhomogeneity, i.e. the varying image intensity, occurs often and it poses considerable challenges for image segmentation. In this paper, we propose an efficient variational method for segmenting images with intensity inhomogeneity. The method is inspired by previo… ▽ More Image segmentation is an important task in the domain of computer vision and medical imaging. In natural and medical images, intensity inhomogeneity, i.e. the varying image intensity, occurs often and it poses considerable challenges for image segmentation. In this paper, we propose an efficient variational method for segmenting images with intensity inhomogeneity. The method is inspired by previous works on two-stage segmentation and variational Retinex. Our method consists of two stages. In the first stage, we decouple the image into reflection and illumination parts by solving a convex energy minimization model with either total variation or tight-frame regularisation. In the second stage, we segment the original image by thresholding on the reflection part, and the inhomogeneous intensity is estimated by the smoothly varying illumination part. We adopt a primal dual algorithm to solve the convex model in the first stage, and the convergence is guaranteed. Numerical experiments clearly show that our method is robust and efficient to segment both natural and medical images. △ Less

Submitted 3 April, 2019; originally announced April 2019.

Wireless Communications and Control for Swarms of Cellular-Connected UAVs

Authors: Tengchan Zeng, Mohammad Mozaffari, Omid Semiari, Walid Saad, Mehdi Bennis, Merouane Debbah

Abstract: By using wireless connectivity through cellular base stations (BSs), swarms of unmanned aerial vehicles (UAVs) can provide a plethora of services ranging from delivery of goods to surveillance. In particular, UAVs in a swarm can utilize wireless communications to collect information, like velocity and heading angle, from surrounding UAVs for coordinating their operations and maintaining target spe… ▽ More By using wireless connectivity through cellular base stations (BSs), swarms of unmanned aerial vehicles (UAVs) can provide a plethora of services ranging from delivery of goods to surveillance. In particular, UAVs in a swarm can utilize wireless communications to collect information, like velocity and heading angle, from surrounding UAVs for coordinating their operations and maintaining target speed and intra-UAV distance. However, due to the uncertainty of the wireless channel, wireless communications among UAVs will experience a transmission delay which can impair the swarm's ability to stabilize system operation. In this paper, the problem of joint communication and control is studied for a swarm of three cellular-connected UAVs positioned in a triangle formation. In particular, a novel approach is proposed for optimizing the swarm's operation while jointly considering the delay of the wireless network and the stability of the control system. Based on this approach, the maximum allowable delay required to prevent the instability of the swarm is determined. Moreover, by using stochastic geometry, the reliability of the wireless network is derived as the probability of meeting the stability requirement of the control system. The simulation results validate the effectiveness of the proposed joint strategy, and help obtain insightful design guidelines on how to form a stable swarm of UAVs. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Advanced Denoising for X-ray Ptychography

Authors: Huibin Chang, Pablo Enfedaque, Jie Zhang, Juliane Reinhardt, Bjoern Enders, Young-Sang Yu, David Shapiro, Christian G. Schroer, Tieyong Zeng, Stefano Marchesini

Abstract: The success of ptychographic imaging experiments strongly depends on achieving high signal-to-noise ratio. This is particularly important in nanoscale imaging experiments when diffraction signals are very weak and the experiments are accompanied by significant parasitic scattering (background), outliers or correlated noise sources. It is also critical when rare events such as cosmic rays, or bad f… ▽ More The success of ptychographic imaging experiments strongly depends on achieving high signal-to-noise ratio. This is particularly important in nanoscale imaging experiments when diffraction signals are very weak and the experiments are accompanied by significant parasitic scattering (background), outliers or correlated noise sources. It is also critical when rare events such as cosmic rays, or bad frames caused by electronic glitches or shutter timing malfunction take place. In this paper, we propose a novel iterative algorithm with rigorous analysis that exploits the direct forward model for parasitic noise and sample smoothness to achieve a thorough characterization and removal of structured and random noise. We present a formal description of the proposed algorithm and prove its convergence under mild conditions. Numerical experiments from simulations and real data (both soft and hard X-ray beamlines) demonstrate that the proposed algorithms produce better results when compared to state-of-the-art methods. △ Less

Submitted 14 February, 2019; v1 submitted 5 November, 2018; originally announced November 2018.

Comments: 24 pages, 9 figures

Journal ref: Optics express 27 (8), 10395-10418 (2019)

Simultaneous identification of linear building dynamic model and disturbance using sparsity-promoting optimization

Authors: Tingting Zeng, Jonathan Brooks, Prabir Barooah

Abstract: We propose a method that simultaneously identifies a linear time-invariant model of a building's temperature dynamics and a transformed version of the unmeasured disturbance affecting the building. Our method uses l1-regularization to encourage the identified disturbance to be approximately sparse, which is motivated by the slowly-varying nature of occupancy that determines the disturbance. The pr… ▽ More We propose a method that simultaneously identifies a linear time-invariant model of a building's temperature dynamics and a transformed version of the unmeasured disturbance affecting the building. Our method uses l1-regularization to encourage the identified disturbance to be approximately sparse, which is motivated by the slowly-varying nature of occupancy that determines the disturbance. The proposed method involves solving a convex optimization problem that guarantees the identified black-box model possesses known properties of the plant, especially input-output stability and positive DC gains. These features enable one to use the method as part of a self-learning control system in which the model of the building is updated periodically without requiring human intervention. Results from the application of the method on data from a simulated and real building are provided. △ Less

Submitted 10 June, 2020; v1 submitted 16 November, 2017; originally announced November 2017.