-
Transformer-based interpretable multi-modal data fusion for skin lesion classification
Authors:
Theodor Cheslerean-Boghiu,
Melia-Evelina Fleischmann,
Theresa Willem,
Tobias Lasser
Abstract:
A lot of deep learning (DL) research these days is mainly focused on improving quantitative metrics regardless of other factors. In human-centered applications, like skin lesion classification in dermatology, DL-driven clinical decision support systems are still in their infancy due to the limited transparency of their decision-making process. Moreover, the lack of procedures that can explain the…
▽ More
A lot of deep learning (DL) research these days is mainly focused on improving quantitative metrics regardless of other factors. In human-centered applications, like skin lesion classification in dermatology, DL-driven clinical decision support systems are still in their infancy due to the limited transparency of their decision-making process. Moreover, the lack of procedures that can explain the behavior of trained DL algorithms leads to almost no trust from clinical physicians. To diagnose skin lesions, dermatologists rely on visual assessment of the disease and the data gathered from the patient's anamnesis. Data-driven algorithms dealing with multi-modal data are limited by the separation of feature-level and decision-level fusion procedures required by convolutional architectures. To address this issue, we enable single-stage multi-modal data fusion via the attention mechanism of transformer-based architectures to aid in diagnosing skin diseases. Our method beats other state-of-the-art single- and multi-modal DL architectures in image-rich and patient-data-rich environments. Additionally, the choice of the architecture enables native interpretability support for the classification task both in the image and metadata domain with no additional modifications necessary.
△ Less
Submitted 31 August, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
-
WNet: A data-driven dual-domain denoising model for sparse-view computed tomography with a trainable reconstruction layer
Authors:
Theodor Cheslerean-Boghiu,
Felix C. Hofmann,
Manuel Schultheiß,
Franz Pfeiffer,
Daniela Pfeiffer,
Tobias Lasser
Abstract:
Deep learning based solutions are being succesfully implemented for a wide variety of applications. Most notably, clinical use-cases have gained an increased interest and have been the main driver behind some of the cutting-edge data-driven algorithms proposed in the last years. For applications like sparse-view tomographic reconstructions, where the amount of measurement data is small in order to…
▽ More
Deep learning based solutions are being succesfully implemented for a wide variety of applications. Most notably, clinical use-cases have gained an increased interest and have been the main driver behind some of the cutting-edge data-driven algorithms proposed in the last years. For applications like sparse-view tomographic reconstructions, where the amount of measurement data is small in order to keep acquisition time short and radiation dose low, reduction of the streaking artifacts has prompted the development of data-driven denoising algorithms with the main goal of obtaining diagnostically viable images with only a subset of a full-scan data. We propose WNet, a data-driven dual-domain denoising model which contains a trainable reconstruction layer for sparse-view artifact denoising. Two encoder-decoder networks perform denoising in both sinogram- and reconstruction-domain simultaneously, while a third layer implementing the Filtered Backprojection algorithm is sandwiched between the first two and takes care of the reconstruction operation. We investigate the performance of the network on sparse-view chest CT scans, and we highlight the added benefit of having a trainable reconstruction layer over the more conventional fixed ones. We train and test our network on two clinically relevant datasets and we compare the obtained results with three different types of sparse-view CT denoising and reconstruction algorithms.
△ Less
Submitted 3 April, 2023; v1 submitted 1 July, 2022;
originally announced July 2022.
-
Task-specific Performance Prediction and Acquisition Optimization for Anisotropic X-ray Dark-field Tomography
Authors:
Theodor Cheslerean-Boghiu,
Franz Pfeiffer,
Tobias Lasser
Abstract:
Anisotropic X-ray Dark-field Tomography (AXDT) is a recently developed imaging modality that enables the visualization of oriented microstructures using lab-based X-ray grating interferometer setups. While there are very promising application scenarios, for example in materials testing of fibrous composites or in medical diagnosis of brain cell connectivity, AXDT faces challenges in practical appl…
▽ More
Anisotropic X-ray Dark-field Tomography (AXDT) is a recently developed imaging modality that enables the visualization of oriented microstructures using lab-based X-ray grating interferometer setups. While there are very promising application scenarios, for example in materials testing of fibrous composites or in medical diagnosis of brain cell connectivity, AXDT faces challenges in practical applicability due to the complex and time-intensive acquisitions required to fully sample the anisotropic X-ray scattering functions. However, depending on the specific imaging task at hand, a full sampling may not be required, allowing for reduced acquisitions. In this work we are investigating a performance prediction approach for AXDT using task-specific detectability indices. Based on this approach we present a task-driven acquisition optimization method that enables reduced acquisition schemes while kee** the task-specific image quality high. We demonstrate the feasibility and efficacy of the method in experiments with simulated and experimental data.
△ Less
Submitted 1 July, 2022;
originally announced July 2022.
