-
Multimodal Learning With Intraoperative CBCT & Variably Aligned Preoperative CT Data To Improve Segmentation
Authors:
Maximilian E. Tschuchnig,
Philipp Steininger,
Michael Gadermayr
Abstract:
Cone-beam computed tomography (CBCT) is an important tool facilitating computer aided interventions, despite often suffering from artifacts that pose challenges for accurate interpretation. While the degraded image quality can affect downstream segmentation, the availability of high quality, preoperative scans represents potential for improvements. Here we consider a setting where preoperative CT…
▽ More
Cone-beam computed tomography (CBCT) is an important tool facilitating computer aided interventions, despite often suffering from artifacts that pose challenges for accurate interpretation. While the degraded image quality can affect downstream segmentation, the availability of high quality, preoperative scans represents potential for improvements. Here we consider a setting where preoperative CT and intraoperative CBCT scans are available, however, the alignment (registration) between the scans is imperfect. We propose a multimodal learning method that fuses roughly aligned CBCT and CT scans and investigate the effect of CBCT quality and misalignment on the final segmentation performance. For that purpose, we make use of a synthetically generated data set containing real CT and synthetic CBCT volumes. As an application scenario, we focus on liver and liver tumor segmentation. We show that the fusion of preoperative CT and simulated, intraoperative CBCT mostly improves segmentation performance (compared to using intraoperative CBCT only) and that even clearly misaligned preoperative data has the potential to improve segmentation performance.
△ Less
Submitted 1 July, 2024; v1 submitted 17 June, 2024;
originally announced June 2024.
-
Multi-task Learning To Improve Semantic Segmentation Of CBCT Scans Using Image Reconstruction
Authors:
Maximilian Ernst Tschuchnig,
Julia Coste-Marin,
Philipp Steininger,
Michael Gadermayr
Abstract:
Semantic segmentation is a crucial task in medical image processing, essential for segmenting organs or lesions such as tumors. In this study we aim to improve automated segmentation in CBCTs through multi-task learning. To evaluate effects on different volume qualities, a CBCT dataset is synthesised from the CT Liver Tumor Segmentation Benchmark (LiTS) dataset. To improve segmentation, two approa…
▽ More
Semantic segmentation is a crucial task in medical image processing, essential for segmenting organs or lesions such as tumors. In this study we aim to improve automated segmentation in CBCTs through multi-task learning. To evaluate effects on different volume qualities, a CBCT dataset is synthesised from the CT Liver Tumor Segmentation Benchmark (LiTS) dataset. To improve segmentation, two approaches are investigated. First, we perform multi-task learning to add morphology based regularization through a volume reconstruction task. Second, we use this reconstruction task to reconstruct the best quality CBCT (most similar to the original CT), facilitating denoising effects. We explore both holistic and patch-based approaches. Our findings reveal that, especially using a patch-based approach, multi-task learning improves segmentation in most cases and that these results can further be improved by our denoising approach.
△ Less
Submitted 20 December, 2023;
originally announced December 2023.
-
Evaluation of Multi-Scale Multiple Instance Learning to Improve Thyroid Cancer Classification
Authors:
Maximilian E. Tschuchnig,
Philipp Grubmüller,
Lea M. Stangassinger,
Christina Kreutzer,
Sébastien Couillard-Després,
Gertie J. Oostingh,
Anton Hittmair,
Michael Gadermayr
Abstract:
Thyroid cancer is currently the fifth most common malignancy diagnosed in women. Since differentiation of cancer sub-types is important for treatment and current, manual methods are time consuming and subjective, automatic computer-aided differentiation of cancer types is crucial. Manual differentiation of thyroid cancer is based on tissue sections, analysed by pathologists using histological feat…
▽ More
Thyroid cancer is currently the fifth most common malignancy diagnosed in women. Since differentiation of cancer sub-types is important for treatment and current, manual methods are time consuming and subjective, automatic computer-aided differentiation of cancer types is crucial. Manual differentiation of thyroid cancer is based on tissue sections, analysed by pathologists using histological features. Due to the enormous size of gigapixel whole slide images, holistic classification using deep learning methods is not feasible. Patch based multiple instance learning approaches, combined with aggregations such as bag-of-words, is a common approach. This work's contribution is to extend a patch based state-of-the-art method by generating and combining feature vectors of three different patch resolutions and analysing three distinct ways of combining them. The results showed improvements in one of the three multi-scale approaches, while the others led to decreased scores. This provides motivation for analysis and discussion of the individual approaches.
△ Less
Submitted 22 April, 2022;
originally announced April 2022.
-
Anomaly Detection in Medical Imaging -- A Mini Review
Authors:
Maximilian E. Tschuchnig,
Michael Gadermayr
Abstract:
The increasing digitization of medical imaging enables machine learning based improvements in detecting, visualizing and segmenting lesions, easing the workload for medical experts. However, supervised machine learning requires reliable labelled data, which is is often difficult or impossible to collect or at least time consuming and thereby costly. Therefore methods requiring only partly labeled…
▽ More
The increasing digitization of medical imaging enables machine learning based improvements in detecting, visualizing and segmenting lesions, easing the workload for medical experts. However, supervised machine learning requires reliable labelled data, which is is often difficult or impossible to collect or at least time consuming and thereby costly. Therefore methods requiring only partly labeled data (semi-supervised) or no labeling at all (unsupervised methods) have been applied more regularly. Anomaly detection is one possible methodology that is able to leverage semi-supervised and unsupervised methods to handle medical imaging tasks like classification and segmentation. This paper uses a semi-exhaustive literature review of relevant anomaly detection papers in medical imaging to cluster into applications, highlight important results, establish lessons learned and give further advice on how to approach anomaly detection in medical imaging. The qualitative analysis is based on google scholar and 4 different search terms, resulting in 120 different analysed papers. The main results showed that the current research is mostly motivated by reducing the need for labelled data. Also, the successful and substantial amount of research in the brain MRI domain shows the potential for applications in further domains like OCT and chest X-ray.
△ Less
Submitted 25 August, 2021;
originally announced August 2021.
-
Frozen-to-Paraffin: Categorization of Histological Frozen Sections by the Aid of Paraffin Sections and Generative Adversarial Networks
Authors:
Michael Gadermayr,
Maximilian Tschuchnig,
Lea Maria Stangassinger,
Christina Kreutzer,
Sebastien Couillard-Despres,
Gertie Janneke Oostingh,
Anton Hittmair
Abstract:
In contrast to paraffin sections, frozen sections can be quickly generated during surgical interventions. This procedure allows surgeons to wait for histological findings during the intervention to base intra-operative decisions on the outcome of the histology. However, compared to paraffin sections, the quality of frozen sections is typically lower, leading to a higher ratio of miss-classificatio…
▽ More
In contrast to paraffin sections, frozen sections can be quickly generated during surgical interventions. This procedure allows surgeons to wait for histological findings during the intervention to base intra-operative decisions on the outcome of the histology. However, compared to paraffin sections, the quality of frozen sections is typically lower, leading to a higher ratio of miss-classification. In this work, we investigated the effect of the section type on automated decision support approaches for classification of thyroid cancer. This was enabled by a data set consisting of pairs of sections for individual patients. Moreover, we investigated, whether a frozen-to-paraffin translation could help to optimize classification scores. Finally, we propose a specific data augmentation strategy to deal with a small amount of training data and to increase classification accuracy even further.
△ Less
Submitted 27 July, 2021; v1 submitted 15 December, 2020;
originally announced December 2020.
-
Generative Adversarial Networks in Digital Pathology: A Survey on Trends and Future Potential
Authors:
Maximilian Ernst Tschuchnig,
Gertie Janneke Oostingh,
Michael Gadermayr
Abstract:
Image analysis in the field of digital pathology has recently gained increased popularity. The use of high-quality whole slide scanners enables the fast acquisition of large amounts of image data, showing extensive context and microscopic detail at the same time. Simultaneously, novel machine learning algorithms have boosted the performance of image analysis approaches. In this paper, we focus on…
▽ More
Image analysis in the field of digital pathology has recently gained increased popularity. The use of high-quality whole slide scanners enables the fast acquisition of large amounts of image data, showing extensive context and microscopic detail at the same time. Simultaneously, novel machine learning algorithms have boosted the performance of image analysis approaches. In this paper, we focus on a particularly powerful class of architectures, called Generative Adversarial Networks (GANs), applied to histological image data. Besides improving performance, GANs also enable application scenarios in this field, which were previously intractable. However, GANs could exhibit a potential for introducing bias. Hereby, we summarize the recent state-of-the-art developments in a generalizing notation, present the main applications of GANs and give an outlook of some chosen promising approaches and their possible future applications. In addition, we identify currently unavailable methods with potential for future applications.
△ Less
Submitted 7 May, 2020; v1 submitted 30 April, 2020;
originally announced April 2020.
-
An Asymmetric Cycle-Consistency Loss for Dealing with Many-to-One Map**s in Image Translation: A Study on Thigh MR Scans
Authors:
Michael Gadermayr,
Maximilian Tschuchnig,
Laxmi Gupta,
Dorit Merhof,
Nils Krämer,
Daniel Truhn,
Burkhard Gess
Abstract:
Generative adversarial networks using a cycle-consistency loss facilitate unpaired training of image-translation models and thereby exhibit a very high potential in manifold medical applications. However, the fact that images in one domain potentially map to more than one image in another domain (e.g. in case of pathological changes) exhibits a major challenge for training the networks. In this wo…
▽ More
Generative adversarial networks using a cycle-consistency loss facilitate unpaired training of image-translation models and thereby exhibit a very high potential in manifold medical applications. However, the fact that images in one domain potentially map to more than one image in another domain (e.g. in case of pathological changes) exhibits a major challenge for training the networks. In this work, we offer a solution to improve the training process in case of many-to-one map**s by modifying the cycle-consistency loss. We show formally and empirically that the proposed method improves the performance significantly without radically changing the architecture and without increasing the overall complexity. We evaluate our method on thigh MRI scans with the final goal of segmenting the muscle in fat-infiltrated patients' data.
△ Less
Submitted 11 January, 2021; v1 submitted 23 April, 2020;
originally announced April 2020.
