-
Self-Supervised Learning from Unlabeled Fundus Photographs Improves Segmentation of the Retina
Authors:
Jan Kukačka,
Anja Zenz,
Marcel Kollovieh,
Dominik Jüstel,
Vasilis Ntziachristos
Abstract:
Fundus photography is the primary method for retinal imaging and essential for diabetic retinopathy prevention. Automated segmentation of fundus photographs would improve the quality, capacity, and cost-effectiveness of eye care screening programs. However, current segmentation methods are not robust towards the diversity in imaging conditions and pathologies typical for real-world clinical applic…
▽ More
Fundus photography is the primary method for retinal imaging and essential for diabetic retinopathy prevention. Automated segmentation of fundus photographs would improve the quality, capacity, and cost-effectiveness of eye care screening programs. However, current segmentation methods are not robust towards the diversity in imaging conditions and pathologies typical for real-world clinical applications. To overcome these limitations, we utilized contrastive self-supervised learning to exploit the large variety of unlabeled fundus images in the publicly available EyePACS dataset. We pre-trained an encoder of a U-Net, which we later fine-tuned on several retinal vessel and lesion segmentation datasets. We demonstrate for the first time that by using contrastive self-supervised learning, the pre-trained network can recognize blood vessels, optic disc, fovea, and various lesions without being provided any labels. Furthermore, when fine-tuned on a downstream blood vessel segmentation task, such pre-trained networks achieve state-of-the-art performance on images from different datasets. Additionally, the pre-training also leads to shorter training times and an improved few-shot performance on both blood vessel and lesion segmentation tasks. Altogether, our results showcase the benefits of contrastive self-supervised pre-training which can play a crucial role in real-world clinical applications requiring robust models able to adapt to new devices with only a few annotated samples.
△ Less
Submitted 5 August, 2021;
originally announced August 2021.
-
Deep learning based electrical noise removal enables high spectral optoacoustic contrast in deep tissue
Authors:
Christoph Dehner,
Ivan Olefir,
Kaushik Basak Chowdhury,
Dominik Jüstel,
Vasilis Ntziachristos
Abstract:
Image contrast in multispectral optoacoustic tomography (MSOT) can be severely reduced by electrical noise and interference in the acquired optoacoustic signals. Signal processing techniques have proven insufficient to remove the effects of electrical noise because they typically rely on simplified models and fail to capture complex characteristics of signal and noise. Moreover, they often involve…
▽ More
Image contrast in multispectral optoacoustic tomography (MSOT) can be severely reduced by electrical noise and interference in the acquired optoacoustic signals. Signal processing techniques have proven insufficient to remove the effects of electrical noise because they typically rely on simplified models and fail to capture complex characteristics of signal and noise. Moreover, they often involve time-consuming processing steps that are unsuited for real-time imaging applications. In this work, we develop and demonstrate a discriminative deep learning (DL) approach to separate electrical noise from optoacoustic signals prior to image reconstruction. The proposed DL algorithm is based on two key features. First, it learns spatiotemporal correlations in both noise and signal by using the entire optoacoustic sinogram as input. Second, it employs training based on a large dataset of experimentally acquired pure noise and synthetic optoacoustic signals. We validated the ability of the trained model to accurately remove electrical noise on synthetic data and on optoacoustic images of a phantom and the human breast. We demonstrate significant enhancements of morphological and spectral optoacoustic images reaching 19% higher blood vessel contrast and localized spectral contrast at depths of more than 2 cm for images acquired in vivo. We discuss how the proposed denoising framework is applicable to clinical multispectral optoacoustic tomography and suitable for real-time operation.
△ Less
Submitted 24 February, 2021;
originally announced February 2021.
-
A distance-based loss for smooth and continuous skin layer segmentation in optoacoustic images
Authors:
Stefan Gerl,
Johannes C. Paetzold,
Hailong He,
Ivan Ezhov,
Suprosanna Shit,
Florian Kofler,
Amirhossein Bayat,
Giles Tetteh,
Vasilis Ntziachristos,
Bjoern Menze
Abstract:
Raster-scan optoacoustic mesoscopy (RSOM) is a powerful, non-invasive optical imaging technique for functional, anatomical, and molecular skin and tissue analysis. However, both the manual and the automated analysis of such images are challenging, because the RSOM images have very low contrast, poor signal to noise ratio, and systematic overlaps between the absorption spectra of melanin and hemogl…
▽ More
Raster-scan optoacoustic mesoscopy (RSOM) is a powerful, non-invasive optical imaging technique for functional, anatomical, and molecular skin and tissue analysis. However, both the manual and the automated analysis of such images are challenging, because the RSOM images have very low contrast, poor signal to noise ratio, and systematic overlaps between the absorption spectra of melanin and hemoglobin. Nonetheless, the segmentation of the epidermis layer is a crucial step for many downstream medical and diagnostic tasks, such as vessel segmentation or monitoring of cancer progression. We propose a novel, shape-specific loss function that overcomes discontinuous segmentations and achieves smooth segmentation surfaces while preserving the same volumetric Dice and IoU. Further, we validate our epidermis segmentation through the sensitivity of vessel segmentation. We found a 20 $\%$ improvement in Dice for vessel segmentation tasks when the epidermis mask is provided as additional information to the vessel segmentation network.
△ Less
Submitted 10 July, 2020;
originally announced July 2020.
-
Maximum entropy based non-negative optoacoustic tomographic image reconstruction
Authors:
Jaya Prakash,
Subhamoy Mandal,
Daniel Razansky,
Vasilis Ntziachristos
Abstract:
Objective:Optoacoustic (photoacoustic) tomography is aimed at reconstructing maps of the initial pressure rise induced by the absorption of light pulses in tissue. In practice, due to inaccurate assumptions in the forward model, noise and other experimental factors, the images are often afflicted by artifacts, occasionally manifested as negative values. The aim of the work is to develop an inversi…
▽ More
Objective:Optoacoustic (photoacoustic) tomography is aimed at reconstructing maps of the initial pressure rise induced by the absorption of light pulses in tissue. In practice, due to inaccurate assumptions in the forward model, noise and other experimental factors, the images are often afflicted by artifacts, occasionally manifested as negative values. The aim of the work is to develop an inversion method which reduces the occurrence of negative values and improves the quantitative performance of optoacoustic imaging. Methods: We present a novel method for optoacoustic tomography based on an entropy maximization algorithm, which uses logarithmic regularization for attaining non-negative reconstructions. The reconstruction image quality is further improved using structural prior based fluence correction. Results: We report the performance achieved by the entropy maximization scheme on numerical simulation, experimental phantoms and in-vivo samples. Conclusion: The proposed algorithm demonstrates superior reconstruction performance by delivering non-negative pixel values with no visible distortion of anatomical structures. Significance: Our method can enable quantitative optoacoustic imaging, and has the potential to improve pre-clinical and translational imaging applications.
△ Less
Submitted 11 January, 2019; v1 submitted 26 July, 2017;
originally announced July 2017.
-
Eigenspectra optoacoustic tomography achieves quantitative blood oxygenation imaging deep in tissues
Authors:
Stratis Tzoumas,
Antonio Nunes,
Ivan Olefir,
Stefan Stangl,
Panagiotis Symvoulidis,
Sarah Glasl,
Christine Bayer,
Gabriele Multhoff,
Vasilis Ntziachristos
Abstract:
Light propagating in tissue attains a spectrum that varies with location due to wavelength-dependent fluence attenuation by tissue optical properties, an effect that causes spectral corruption. Predictions of the spectral variations of light fluence in tissue are challenging since the spatial distribution of optical properties in tissue cannot be resolved in high resolution or with high accuracy b…
▽ More
Light propagating in tissue attains a spectrum that varies with location due to wavelength-dependent fluence attenuation by tissue optical properties, an effect that causes spectral corruption. Predictions of the spectral variations of light fluence in tissue are challenging since the spatial distribution of optical properties in tissue cannot be resolved in high resolution or with high accuracy by current methods. Spectral corruption has fundamentally limited the quantification accuracy of optical and optoacoustic methods and impeded the long sought-after goal of imaging blood oxygen saturation (sO2) deep in tissues; a critical but still unattainable target for the assessment of oxygenation in physiological processes and disease. We discover a new principle underlying light fluence in tissues, which describes the wavelength dependence of light fluence as an affine function of a few reference base spectra, independently of the specific distribution of tissue optical properties. This finding enables the introduction of a previously undocumented concept termed eigenspectra Multispectral Optoacoustic Tomography (eMSOT) that can effectively account for wavelength dependent light attenuation without explicit knowledge of the tissue optical properties. We validate eMSOT in more than 2000 simulations and with phantom and animal measurements. We find that eMSOT can quantitatively image tissue sO2 reaching in many occasions a better than 10-fold improved accuracy over conventional spectral optoacoustic methods. Then, we show that eMSOT can spatially resolve sO2 in muscle and tumor; revealing so far unattainable tissue physiology patterns. Last, we related eMSOT readings to cancer hypoxia and found congruence between eMSOT tumor sO2 images and tissue perfusion and hypoxia maps obtained by correlative histological analysis.
△ Less
Submitted 18 November, 2015;
originally announced November 2015.
