-
Self-supervised learning for classifying paranasal anomalies in the maxillary sinus
Authors:
Debayan Bhattacharya,
Finn Behrendt,
Benjamin Tobias Becker,
Lennart Maack,
Dirk Beyersdorff,
Elina Petersen,
Marvin Petersen,
Bastian Cheng,
Dennis Eggert,
Christian Betz,
Anna Sophie Hoffmann,
Alexander Schlaefer
Abstract:
Purpose: Paranasal anomalies, frequently identified in routine radiological screenings, exhibit diverse morphological characteristics. Due to the diversity of anomalies, supervised learning methods require large labelled dataset exhibiting diverse anomaly morphology. Self-supervised learning (SSL) can be used to learn representations from unlabelled data. However, there are no SSL methods designed…
▽ More
Purpose: Paranasal anomalies, frequently identified in routine radiological screenings, exhibit diverse morphological characteristics. Due to the diversity of anomalies, supervised learning methods require large labelled dataset exhibiting diverse anomaly morphology. Self-supervised learning (SSL) can be used to learn representations from unlabelled data. However, there are no SSL methods designed for the downstream task of classifying paranasal anomalies in the maxillary sinus (MS).
Methods: Our approach uses a 3D Convolutional Autoencoder (CAE) trained in an unsupervised anomaly detection (UAD) framework. Initially, we train the 3D CAE to reduce reconstruction errors when reconstructing normal maxillary sinus (MS) image. Then, this CAE is applied to an unlabelled dataset to generate coarse anomaly locations by creating residual MS images. Following this, a 3D Convolutional Neural Network (CNN) reconstructs these residual images, which forms our SSL task. Lastly, we fine-tune the encoder part of the 3D CNN on a labelled dataset of normal and anomalous MS images.
Results: The proposed SSL technique exhibits superior performance compared to existing generic self-supervised methods, especially in scenarios with limited annotated data. When trained on just 10% of the annotated dataset, our method achieves an Area Under the Precision-Recall Curve (AUPRC) of 0.79 for the downstream classification task. This performance surpasses other methods, with BYOL attaining an AUPRC of 0.75, SimSiam at 0.74, SimCLR at 0.73 and Masked Autoencoding using SparK at 0.75.
Conclusion: A self-supervised learning approach that inherently focuses on localizing paranasal anomalies proves to be advantageous, particularly when the subsequent task involves differentiating normal from anomalous maxillary sinuses. Access our code at https://github.com/mtec-tuhh/self-supervised-paranasal-anomaly
△ Less
Submitted 29 April, 2024;
originally announced April 2024.
-
Diffusion Models with Ensembled Structure-Based Anomaly Scoring for Unsupervised Anomaly Detection
Authors:
Finn Behrendt,
Debayan Bhattacharya,
Lennart Maack,
Julia Krüger,
Roland Opfer,
Robin Mieling,
Alexander Schlaefer
Abstract:
Supervised deep learning techniques show promise in medical image analysis. However, they require comprehensive annotated data sets, which poses challenges, particularly for rare diseases. Consequently, unsupervised anomaly detection (UAD) emerges as a viable alternative for pathology segmentation, as only healthy data is required for training. However, recent UAD anomaly scoring functions often f…
▽ More
Supervised deep learning techniques show promise in medical image analysis. However, they require comprehensive annotated data sets, which poses challenges, particularly for rare diseases. Consequently, unsupervised anomaly detection (UAD) emerges as a viable alternative for pathology segmentation, as only healthy data is required for training. However, recent UAD anomaly scoring functions often focus on intensity only and neglect structural differences, which impedes the segmentation performance. This work investigates the potential of Structural Similarity (SSIM) to bridge this gap. SSIM captures both intensity and structural disparities and can be advantageous over the classical $l1$ error. However, we show that there is more than one optimal kernel size for the SSIM calculation for different pathologies. Therefore, we investigate an adaptive ensembling strategy for various kernel sizes to offer a more pathology-agnostic scoring mechanism. We demonstrate that this ensembling strategy can enhance the performance of DMs and mitigate the sensitivity to different kernel sizes across varying pathologies, highlighting its promise for brain MRI anomaly detection.
△ Less
Submitted 21 March, 2024;
originally announced March 2024.
-
PolypNextLSTM: A lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM
Authors:
Debayan Bhattacharya,
Konrad Reuter,
Finn Behrendt,
Lennart Maack,
Sarah Grube,
Alexander Schlaefer
Abstract:
Commonly employed in polyp segmentation, single image UNet architectures lack the temporal insight clinicians gain from video data in diagnosing polyps. To mirror clinical practices more faithfully, our proposed solution, PolypNextLSTM, leverages video-based deep learning, harnessing temporal information for superior segmentation performance with the least parameter overhead, making it possibly su…
▽ More
Commonly employed in polyp segmentation, single image UNet architectures lack the temporal insight clinicians gain from video data in diagnosing polyps. To mirror clinical practices more faithfully, our proposed solution, PolypNextLSTM, leverages video-based deep learning, harnessing temporal information for superior segmentation performance with the least parameter overhead, making it possibly suitable for edge devices. PolypNextLSTM employs a UNet-like structure with ConvNext-Tiny as its backbone, strategically omitting the last two layers to reduce parameter overhead. Our temporal fusion module, a Convolutional Long Short Term Memory (ConvLSTM), effectively exploits temporal features. Our primary novelty lies in PolypNextLSTM, which stands out as the leanest in parameters and the fastest model, surpassing the performance of five state-of-the-art image and video-based deep learning models. The evaluation of the SUN-SEG dataset spans easy-to-detect and hard-to-detect polyp scenarios, along with videos containing challenging artefacts like fast motion and occlusion. Comparison against 5 image-based and 5 video-based models demonstrates PolypNextLSTM's superiority, achieving a Dice score of 0.7898 on the hard-to-detect polyp test set, surpassing image-based PraNet (0.7519) and video-based PNSPlusNet (0.7486). Notably, our model excels in videos featuring complex artefacts such as ghosting and occlusion. PolypNextLSTM, integrating pruned ConvNext-Tiny with ConvLSTM for temporal fusion, not only exhibits superior segmentation performance but also maintains the highest frames per speed among evaluated models. Access code here https://github.com/mtec-tuhh/PolypNextLSTM
△ Less
Submitted 28 February, 2024; v1 submitted 18 February, 2024;
originally announced February 2024.
-
Guided Reconstruction with Conditioned Diffusion Models for Unsupervised Anomaly Detection in Brain MRIs
Authors:
Finn Behrendt,
Debayan Bhattacharya,
Robin Mieling,
Lennart Maack,
Julia Krüger,
Roland Opfer,
Alexander Schlaefer
Abstract:
Unsupervised anomaly detection in Brain MRIs aims to identify abnormalities as outliers from a healthy training distribution. Reconstruction-based approaches that use generative models to learn to reconstruct healthy brain anatomy are commonly used for this task. Diffusion models are an emerging class of deep generative models that show great potential regarding reconstruction fidelity. However, t…
▽ More
Unsupervised anomaly detection in Brain MRIs aims to identify abnormalities as outliers from a healthy training distribution. Reconstruction-based approaches that use generative models to learn to reconstruct healthy brain anatomy are commonly used for this task. Diffusion models are an emerging class of deep generative models that show great potential regarding reconstruction fidelity. However, they face challenges in preserving intensity characteristics in the reconstructed images, limiting their performance in anomaly detection. To address this challenge, we propose to condition the denoising mechanism of diffusion models with additional information about the image to reconstruct coming from a latent representation of the noise-free input image. This conditioning enables high-fidelity reconstruction of healthy brain structures while aligning local intensity characteristics of input-reconstruction pairs. We evaluate our method's reconstruction quality, domain adaptation features and finally segmentation performance on publicly available data sets with various pathologies. Using our proposed conditioning mechanism we can reduce the false-positive predictions and enable a more precise delineation of anomalies which significantly enhances the anomaly detection performance compared to established state-of-the-art approaches to unsupervised anomaly detection in brain MRI. Furthermore, our approach shows promise in domain adaptation across different MRI acquisitions and simulated contrasts, a crucial property of general anomaly detection methods.
△ Less
Submitted 7 December, 2023;
originally announced December 2023.
-
Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion
Authors:
Victor Dhédin,
Haolong Li,
Shahram Khorshidi,
Lukas Mack,
Adithya Kumar Chinnakkonda Ravi,
Avadesh Meduri,
Paarth Shah,
Felix Grimminger,
Ludovic Righetti,
Majid Khadiv,
Joerg Stueckler
Abstract:
Implementing dynamic locomotion behaviors on legged robots requires a high-quality state estimation module. Especially when the motion includes flight phases, state-of-the-art approaches fail to produce reliable estimation of the robot posture, in particular base height. In this paper, we propose a novel approach for combining visual-inertial odometry (VIO) with leg odometry in an extended Kalman…
▽ More
Implementing dynamic locomotion behaviors on legged robots requires a high-quality state estimation module. Especially when the motion includes flight phases, state-of-the-art approaches fail to produce reliable estimation of the robot posture, in particular base height. In this paper, we propose a novel approach for combining visual-inertial odometry (VIO) with leg odometry in an extended Kalman filter (EKF) based state estimator. The VIO module uses a stereo camera and IMU to yield low-drift 3D position and yaw orientation and drift-free pitch and roll orientation of the robot base link in the inertial frame. However, these values have a considerable amount of latency due to image processing and optimization, while the rate of update is quite low which is not suitable for low-level control. To reduce the latency, we predict the VIO state estimate at the rate of the IMU measurements of the VIO sensor. The EKF module uses the base pose and linear velocity predicted by VIO, fuses them further with a second high-rate IMU and leg odometry measurements, and produces robot state estimates with a high frequency and small latency suitable for control. We integrate this lightweight estimation framework with a nonlinear model predictive controller and show successful implementation of a set of agile locomotion behaviors, including trotting and jum** at varying horizontal speeds, on a torque-controlled quadruped robot.
△ Less
Submitted 10 October, 2022; v1 submitted 5 October, 2022;
originally announced October 2022.
-
Identifying atmospheric fronts based on diabatic processes using the dynamic state index (DSI)
Authors:
Laura Mack,
Annette Rudolph,
Peter Névir
Abstract:
Atmospheric fronts are associated with precipitation and strong diabatic processes. Therefore, detecting fronts objectively from reanalyses is a prerequisite for the long-term study of their weather impacts. For this purpose, several algorithms exist, e.g., based on the thermic front parameter (TFP) or the F diagnostic. It is shown that both methods have problems to identify weak warm fronts since…
▽ More
Atmospheric fronts are associated with precipitation and strong diabatic processes. Therefore, detecting fronts objectively from reanalyses is a prerequisite for the long-term study of their weather impacts. For this purpose, several algorithms exist, e.g., based on the thermic front parameter (TFP) or the F diagnostic. It is shown that both methods have problems to identify weak warm fronts since they are characterized by low baroclinicity. To avoid this inaccuracy, a new algorithm is developed that considers fronts as deviation from an adiabatic and steady state. These deviations can be accurately measured using the dynamic state index (DSI). The DSI shows a coherent dipole structure along fronts and is strongly correlated with precipitation sums. It is shown that the North Atlantic storm tracks can be clearly identified by the DSI method. Compared to other front identification methods, fronts identified with the DSI method have particularly high specific humidity. Using a simple estimate for front speed, it is shown that fronts identified using DSI method move faster then fronts identified with TFP method, demonstrating the potential of the DSI to indicate movement speed and direction in atmospheric flows.
△ Less
Submitted 24 August, 2022;
originally announced August 2022.
-
Decoupling absorption and emission processes in super-resolution localization of emitters in a plasmonic hotspot
Authors:
David L. Mack,
Emiliano Cortes,
Vincenzo Giannini,
Peter Torok,
Tyler Roschuk,
Stefan A. Maier
Abstract:
The absorption process of an emitter close to a plasmonic antenna is enhanced due to strong local electromagnetic (EM) fields. The emission process, if resonant with the plasmonic system, re-radiates to the far-field by coupling with the antenna due to the availability of plasmonic states. This increases the local density of states (LDOS), effectively providing more, or alternate, pathways for emi…
▽ More
The absorption process of an emitter close to a plasmonic antenna is enhanced due to strong local electromagnetic (EM) fields. The emission process, if resonant with the plasmonic system, re-radiates to the far-field by coupling with the antenna due to the availability of plasmonic states. This increases the local density of states (LDOS), effectively providing more, or alternate, pathways for emission. Through the map** of localized emission events from single molecules close to plasmonic antennas, performed using far-field data, one gains combined information on both the local EM field strength and the LDOS available. The localization from these emission-coupled events generally do not, therefore, report the real position of the molecules, nor the EM enhancement distribution at the illuminating wavelength. Here we propose the use of a large Stokes shift fluorescent molecule in order to spectrally decouple the emission process of the dye from the plasmonic system, leaving only the absorption strongly in resonance with the enhanced EM field in the antennas vicinity. We show that the real position of the emitters in this complex, but interesting, scenario can be found directly. Moreover, we demonstrate that this technique provides an effective way of exploring either the EM field or the LDOS with nanometre spatial resolution.
△ Less
Submitted 3 March, 2017; v1 submitted 17 November, 2016;
originally announced November 2016.
-
Degree-correlation, robustness, and vulnerability in finite scale-free networks
Authors:
Jeremy F. Alm,
Keenan M. L. Mack
Abstract:
Many naturally occurring networks have a power-law degree distribution as well as a non-zero degree correlation. Despite this, most studies analyzing the robustness to random node-deletion and vulnerability to targeted node-deletion have concentrated only on power-law degree distribution and ignored degree correlation. This study looks specifically at the effect degree-correlation has on robustnes…
▽ More
Many naturally occurring networks have a power-law degree distribution as well as a non-zero degree correlation. Despite this, most studies analyzing the robustness to random node-deletion and vulnerability to targeted node-deletion have concentrated only on power-law degree distribution and ignored degree correlation. This study looks specifically at the effect degree-correlation has on robustness and vulnerability in scale-free networks. Our results confirm Newman's finding that positive degree-correlation increases robustness and decreases vulnerability. However, we found that networks with positive degree-correlation are more vulnerable to random node-deletion than to targeted deletion methods that utilize knowledge of initial node-degree only. Targeted deletion sufficiently alters the topology of the network to render this method less effective than uniform random methods unless changes in topology are accounted for. This result indicates the importance of degree correlation in certain network applications.
△ Less
Submitted 15 February, 2017; v1 submitted 28 June, 2016;
originally announced June 2016.
-
Spatial population expansion promotes the evolution of cooperation in an experimental Prisoner's Dilemma
Authors:
J. David Van Dyken,
Melanie J. I. Muller,
Keenan M. L. Mack,
Michael M. Desai
Abstract:
Cooperation is ubiquitous in nature, but explaining its existence remains a central interdisciplinary challenge. Cooperation is most difficult to explain in the Prisoner's Dilemma game, where cooperators always lose in direct competition with defectors despite increasing mean fitness. Here we demonstrate how spatial population expansion, a widespread natural phenomenon, promotes the evolution of c…
▽ More
Cooperation is ubiquitous in nature, but explaining its existence remains a central interdisciplinary challenge. Cooperation is most difficult to explain in the Prisoner's Dilemma game, where cooperators always lose in direct competition with defectors despite increasing mean fitness. Here we demonstrate how spatial population expansion, a widespread natural phenomenon, promotes the evolution of cooperation. We engineer an experimental Prisoner's Dilemma game in the budding yeast Saccharomyces cerevisiae to show that, despite losing to defectors in nonexpanding conditions, cooperators increase in frequency in spatially expanding populations. Fluorescently labeled colonies show genetic demixing of cooperators and defectors, followed by increase in cooperator frequency as cooperator sectors overtake neighboring defector sectors. Together with lattice-based spatial simulations, our results suggest that spatial population expansion drives the evolution of cooperation by (1) increasing positive genetic assortment at population frontiers and (2) selecting for phenotypes maximizing local deme productivity. Spatial expansion thus creates a selective force whereby cooperator-enriched demes overtake neighboring defector-enriched demes in a "survival of the fastest". We conclude that colony growth alone can promote cooperation and prevent defection in microbes. Our results extend to other species with spatially restricted dispersal undergoing range expansion, including pathogens, invasive species, and humans.
△ Less
Submitted 11 November, 2013;
originally announced November 2013.
