-
LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusion
Authors:
Tong Chen,
Qingcheng Lyu,
Long Bai,
Erjian Guo,
Huxin Gao,
Xiaoxiao Yang,
Hongliang Ren,
Lu** Zhou
Abstract:
Advances in endoscopy use in surgeries face challenges like inadequate lighting. Deep learning, notably the Denoising Diffusion Probabilistic Model (DDPM), holds promise for low-light image enhancement in the medical field. However, DDPMs are computationally demanding and slow, limiting their practical medical applications. To bridge this gap, we propose a lightweight DDPM, dubbed LighTDiff. It ad…
▽ More
Advances in endoscopy use in surgeries face challenges like inadequate lighting. Deep learning, notably the Denoising Diffusion Probabilistic Model (DDPM), holds promise for low-light image enhancement in the medical field. However, DDPMs are computationally demanding and slow, limiting their practical medical applications. To bridge this gap, we propose a lightweight DDPM, dubbed LighTDiff. It adopts a T-shape model architecture to capture global structural information using low-resolution images and gradually recover the details in subsequent denoising steps. We further prone the model to significantly reduce the model size while retaining performance. While discarding certain downsampling operations to save parameters leads to instability and low efficiency in convergence during the training, we introduce a Temporal Light Unit (TLU), a plug-and-play module, for more stable training and better performance. TLU associates time steps with denoised image features, establishing temporal dependencies of the denoising steps and improving denoising outcomes. Moreover, while recovering images using the diffusion model, potential spectral shifts were noted. We further introduce a Chroma Balancer (CB) to mitigate this issue. Our LighTDiff outperforms many competitive LLIE methods with exceptional computational efficiency.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Bridging Synthetic and Real Images: a Transferable and Multiple Consistency aided Fundus Image Enhancement Framework
Authors:
Erjian Guo,
Huazhu Fu,
Lu** Zhou,
Dong Xu
Abstract:
Deep learning based image enhancement models have largely improved the readability of fundus images in order to decrease the uncertainty of clinical observations and the risk of misdiagnosis. However, due to the difficulty of acquiring paired real fundus images at different qualities, most existing methods have to adopt synthetic image pairs as training data. The domain shift between the synthetic…
▽ More
Deep learning based image enhancement models have largely improved the readability of fundus images in order to decrease the uncertainty of clinical observations and the risk of misdiagnosis. However, due to the difficulty of acquiring paired real fundus images at different qualities, most existing methods have to adopt synthetic image pairs as training data. The domain shift between the synthetic and the real images inevitably hinders the generalization of such models on clinical data. In this work, we propose an end-to-end optimized teacher-student framework to simultaneously conduct image enhancement and domain adaptation. The student network uses synthetic pairs for supervised enhancement, and regularizes the enhancement model to reduce domain-shift by enforcing teacher-student prediction consistency on the real fundus images without relying on enhanced ground-truth. Moreover, we also propose a novel multi-stage multi-attention guided enhancement network (MAGE-Net) as the backbones of our teacher and student network. Our MAGE-Net utilizes multi-stage enhancement module and retinal structure preservation module to progressively integrate the multi-scale features and simultaneously preserve the retinal structures for better fundus image quality enhancement. Comprehensive experiments on both real and synthetic datasets demonstrate that our framework outperforms the baseline approaches. Moreover, our method also benefits the downstream clinical tasks.
△ Less
Submitted 23 February, 2023;
originally announced February 2023.
-
Synchronous locating and imaging behind scattering medium in a large depth based on deep learning
Authors:
Shuo Zhu,
Enlai Guo,
Qianying Cui,
Dongliang Zheng,
Lianfa Bai,
**g Han
Abstract:
Scattering medium brings great difficulties to locate and image planar objects especially when the object has a large depth. In this letter, a novel learning-based method is presented to locate and image the object hidden behind a thin scattering diffuser. A multi-task network, named DINet, is constructed to predict the depth and the image of the hidden object from the captured speckle patterns. T…
▽ More
Scattering medium brings great difficulties to locate and image planar objects especially when the object has a large depth. In this letter, a novel learning-based method is presented to locate and image the object hidden behind a thin scattering diffuser. A multi-task network, named DINet, is constructed to predict the depth and the image of the hidden object from the captured speckle patterns. The provided experiments verify that the proposed method enables to locate the object with a depth mean error less than 0.05 mm, and image the object with an average PSNR above 24 dB, in a large depth ranging from 350 mm to 1150 mm. The constructed DINet can obtain multiple physical information via a single speckle pattern, including both the depth and image. Comparing with the traditional methods, it paves the way to the practical applications requiring large imaging depth of field behind scattering media.
△ Less
Submitted 29 May, 2020; v1 submitted 25 October, 2019;
originally announced October 2019.
-
Learning-based real-time method to looking through scattering medium beyond the memory effect
Authors:
Enlai Guo,
Shuo Zhu,
Yan Sun,
Lianfa Bai,
**g Han
Abstract:
Strong scattering medium brings great difficulties to optical imaging, which is also a problem in medical imaging and many other fields. Optical memory effect makes it possible to image through strong random scattering medium. However, this method also has the limitation of limited angle field-of-view (FOV), which prevents it from being applied in practice. In this paper, a kind of practical convo…
▽ More
Strong scattering medium brings great difficulties to optical imaging, which is also a problem in medical imaging and many other fields. Optical memory effect makes it possible to image through strong random scattering medium. However, this method also has the limitation of limited angle field-of-view (FOV), which prevents it from being applied in practice. In this paper, a kind of practical convolutional neural network called PDSNet is proposed, which effectively breaks through the limitation of optical memory effect on FOV. Experiments is conducted to prove that the scattered pattern can be reconstructed accurately in real-time by PDSNet, and it is widely applicable to retrieve complex objects of random scales and different scattering media.
△ Less
Submitted 4 November, 2019; v1 submitted 19 October, 2019;
originally announced October 2019.
