-
Interpretable and Interactive Deep Multiple Instance Learning for Dental Caries Classification in Bitewing X-rays
Authors:
Benjamin Bergner,
Csaba Rohrer,
Aiham Taleb,
Martha Duchrau,
Guilherme De Leon,
Jonas Almeida Rodrigues,
Falk Schwendicke,
Joachim Krois,
Christoph Lippert
Abstract:
We propose a simple and efficient image classification architecture based on deep multiple instance learning, and apply it to the challenging task of caries detection in dental radiographs. Technically, our approach contributes in two ways: First, it outputs a heatmap of local patch classification probabilities despite being trained with weak image-level labels. Second, it is amenable to learning…
▽ More
We propose a simple and efficient image classification architecture based on deep multiple instance learning, and apply it to the challenging task of caries detection in dental radiographs. Technically, our approach contributes in two ways: First, it outputs a heatmap of local patch classification probabilities despite being trained with weak image-level labels. Second, it is amenable to learning from segmentation labels to guide training. In contrast to existing methods, the human user can faithfully interpret predictions and interact with the model to decide which regions to attend to. Experiments are conducted on a large clinical dataset of $\sim$38k bitewings ($\sim$316k teeth), where we achieve competitive performance compared to various baselines. When guided by an external caries segmentation model, a significant improvement in classification and localization performance is observed.
△ Less
Submitted 26 September, 2023; v1 submitted 17 December, 2021;
originally announced December 2021.
-
ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics
Authors:
Aiham Taleb,
Matthias Kirchler,
Remo Monti,
Christoph Lippert
Abstract:
High annotation costs are a substantial bottleneck in applying modern deep learning architectures to clinically relevant medical use cases, substantiating the need for novel algorithms to learn from unlabeled data. In this work, we propose ContIG, a self-supervised method that can learn from large datasets of unlabeled medical images and genetic data. Our approach aligns images and several genetic…
▽ More
High annotation costs are a substantial bottleneck in applying modern deep learning architectures to clinically relevant medical use cases, substantiating the need for novel algorithms to learn from unlabeled data. In this work, we propose ContIG, a self-supervised method that can learn from large datasets of unlabeled medical images and genetic data. Our approach aligns images and several genetic modalities in the feature space using a contrastive loss. We design our method to integrate multiple modalities of each individual person in the same model end-to-end, even when the available modalities vary across individuals. Our procedure outperforms state-of-the-art self-supervised methods on all evaluated downstream benchmark tasks. We also adapt gradient-based explainability algorithms to better understand the learned cross-modal associations between the images and genetic modalities. Finally, we perform genome-wide association studies on the features learned by our models, uncovering interesting relationships between images and genetic data.
△ Less
Submitted 26 November, 2021;
originally announced November 2021.
-
Self-Supervised Learning for 3D Medical Image Analysis using 3D SimCLR and Monte Carlo Dropout
Authors:
Yamen Ali,
Aiham Taleb,
Marina M. -C. Höhne,
Christoph Lippert
Abstract:
Self-supervised learning methods can be used to learn meaningful representations from unlabeled data that can be transferred to supervised downstream tasks to reduce the need for labeled data. In this paper, we propose a 3D self-supervised method that is based on the contrastive (SimCLR) method. Additionally, we show that employing Bayesian neural networks (with Monte-Carlo Dropout) during the inf…
▽ More
Self-supervised learning methods can be used to learn meaningful representations from unlabeled data that can be transferred to supervised downstream tasks to reduce the need for labeled data. In this paper, we propose a 3D self-supervised method that is based on the contrastive (SimCLR) method. Additionally, we show that employing Bayesian neural networks (with Monte-Carlo Dropout) during the inference phase can further enhance the results on the downstream tasks. We showcase our models on two medical imaging segmentation tasks: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segmentation from 3D CT. Our experimental results demonstrate the benefits of our proposed methods in both downstream data-efficiency and performance.
△ Less
Submitted 1 October, 2021; v1 submitted 29 September, 2021;
originally announced September 2021.
-
3D Self-Supervised Methods for Medical Imaging
Authors:
Aiham Taleb,
Winfried Loetzsch,
Noel Danz,
Julius Severin,
Thomas Gaertner,
Benjamin Bergner,
Christoph Lippert
Abstract:
Self-supervised learning methods have witnessed a recent surge of interest after proving successful in multiple application fields. In this work, we leverage these techniques, and we propose 3D versions for five different self-supervised methods, in the form of proxy tasks. Our methods facilitate neural network feature learning from unlabeled 3D images, aiming to reduce the required cost for exper…
▽ More
Self-supervised learning methods have witnessed a recent surge of interest after proving successful in multiple application fields. In this work, we leverage these techniques, and we propose 3D versions for five different self-supervised methods, in the form of proxy tasks. Our methods facilitate neural network feature learning from unlabeled 3D images, aiming to reduce the required cost for expert annotation. The developed algorithms are 3D Contrastive Predictive Coding, 3D Rotation prediction, 3D Jigsaw puzzles, Relative 3D patch location, and 3D Exemplar networks. Our experiments show that pretraining models with our 3D tasks yields more powerful semantic representations, and enables solving downstream tasks more accurately and efficiently, compared to training the models from scratch and to pretraining them on 2D slices. We demonstrate the effectiveness of our methods on three downstream tasks from the medical imaging domain: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segmentation from 3D CT, and iii) Diabetic Retinopathy Detection from 2D Fundus images. In each task, we assess the gains in data-efficiency, performance, and speed of convergence. Interestingly, we also find gains when transferring the learned representations, by our methods, from a large unlabeled 3D corpus to a small downstream-specific dataset. We achieve results competitive to state-of-the-art solutions at a fraction of the computational expense. We publish our implementations for the developed algorithms (both 3D and 2D versions) as an open-source library, in an effort to allow other researchers to apply and extend our methods on their datasets.
△ Less
Submitted 2 November, 2020; v1 submitted 6 June, 2020;
originally announced June 2020.
-
Multimodal Self-Supervised Learning for Medical Image Analysis
Authors:
Aiham Taleb,
Christoph Lippert,
Tassilo Klein,
Moin Nabi
Abstract:
Self-supervised learning approaches leverage unlabeled samples to acquire generic knowledge about different concepts, hence allowing for annotation-efficient downstream task learning. In this paper, we propose a novel self-supervised method that leverages multiple imaging modalities. We introduce the multimodal puzzle task, which facilitates rich representation learning from multiple image modalit…
▽ More
Self-supervised learning approaches leverage unlabeled samples to acquire generic knowledge about different concepts, hence allowing for annotation-efficient downstream task learning. In this paper, we propose a novel self-supervised method that leverages multiple imaging modalities. We introduce the multimodal puzzle task, which facilitates rich representation learning from multiple image modalities. The learned representations allow for subsequent fine-tuning on different downstream tasks. To achieve that, we learn a modality-agnostic feature embedding by confusing image modalities at the data-level. Together with the Sinkhorn operator, with which we formulate the puzzle solving optimization as permutation matrix inference instead of classification, they allow for efficient solving of multimodal puzzles with varying levels of complexity. In addition, we also propose to utilize cross-modal generation techniques for multimodal data augmentation used for training self-supervised tasks. In other words, we exploit synthetic images for self-supervised pretraining, instead of downstream tasks directly, in order to circumvent quality issues associated with synthetic images, while improving data-efficiency and representations quality. Our experimental results, which assess the gains in downstream performance and data-efficiency, show that solving our multimodal puzzles yields better semantic representations, compared to treating each modality independently. Our results also highlight the benefits of exploiting synthetic images for self-supervised pretraining. We showcase our approach on four downstream tasks: Brain tumor segmentation and survival days prediction using four MRI modalities, Prostate segmentation using two MRI modalities, and Liver segmentation using unregistered CT and MRI modalities. We outperform many previous solutions, and achieve results competitive to state-of-the-art.
△ Less
Submitted 25 October, 2020; v1 submitted 11 December, 2019;
originally announced December 2019.
-
An imporved decentralized approach for tracking multiple mobile targets through ZigBee WSNs
Authors:
Tareq Alhmiedat,
Amer O. Abu Salem,
Anas Abu Taleb
Abstract:
Target localization and tracking problems in WSNs have received considerable attention recently, driven by the requirement to achieve high localization accuracy, with the minimum cost possible. In WSN based tracking applications, it is critical to know the current location of any sensor node with the minimum energy consumed. This paper focuses on the energy consumption issue in terms of communicat…
▽ More
Target localization and tracking problems in WSNs have received considerable attention recently, driven by the requirement to achieve high localization accuracy, with the minimum cost possible. In WSN based tracking applications, it is critical to know the current location of any sensor node with the minimum energy consumed. This paper focuses on the energy consumption issue in terms of communication between nodes whenever the localization information is transmitted to a sink node. Tracking through WSNs can be categorized into centralized and decentralized systems. Decentralized systems offer low power consumption when deployed to track a small number of mobile targets compared to the centralized tracking systems. However, in several applications, it is essential to position a large number of mobile targets. In such applications, decentralized systems offer high power consumption, since the location of each mobile target is required to be transmitted to a sink node, and this increases the power consumption for the whole WSN. In this paper, we propose a power efficient decentralized approach for tracking a large number of mobile targets while offering reasonable localization accuracy through ZigBee network.
△ Less
Submitted 11 July, 2013;
originally announced July 2013.
