Showing 1–22 of 22 results for author: Berg, A

Time-efficient, high-resolution 3T whole-brain relaxometry using Cartesian 3D MR-STAT with CSF suppression

Authors: Hongyan Liu, Edwin Versteeg, Miha Fuderer, Oscar van der Heide, Martin B. Schilder, Cornelis A. T. van den Berg, Alessandro Sbrizzi

Abstract: Purpose: Current 3D Magnetic Resonance Spin TomogrAphy in Time-domain (MR-STAT) protocols use transient-state, gradient-spoiled gradient-echo sequences that are prone to cerebrospinal fluid (CSF) pulsation artifacts when applied to the brain. This study aims at develo** a 3D MR-STAT protocol for whole-brain relaxometry that overcomes the challenges posed by CSF-induced ghosting artifacts. Method… ▽ More Purpose: Current 3D Magnetic Resonance Spin TomogrAphy in Time-domain (MR-STAT) protocols use transient-state, gradient-spoiled gradient-echo sequences that are prone to cerebrospinal fluid (CSF) pulsation artifacts when applied to the brain. This study aims at develo** a 3D MR-STAT protocol for whole-brain relaxometry that overcomes the challenges posed by CSF-induced ghosting artifacts. Method: We optimized the flip-angle train within the Cartesian 3D MR-STAT framework to achieve two objectives: (1) minimization of the noise level in the reconstructed quantitative maps, and (2) reduction of the CSF-to-white-matter signal ratio to suppress CSF signal and the associated pulsation artifacts. The optimized new sequence was tested on a gel/water-phantom to evaluate the accuracy of the quantitative maps, and on healthy volunteers to explore the effectiveness of the CSF artifact suppression and robustness of the new protocol. Results: A new optimized sequence with both high parameter encoding capability and low CSF intensity was proposed and initially validated in the gel/water-phantom experiment. From in-vivo experiments with five volunteers, the proposed CSF-suppressed sequence shows no CSF ghosting artifacts and overall greatly improved image quality for all quantitative maps compared to the baseline sequence. Statistical analysis indicated low inter-subject and inter-scan variability for quantitative parameters in gray matter and white matter (1.6%-2.4% for T1 and 2.0%-4.6% for T2), demonstrating the robustness of the new sequence. Conclusion: We presented a new 3D MR-STAT sequence with CSF suppression that effectively eliminates CSF pulsation artifacts. The new sequence ensures consistently high-quality, 1mm^3 whole-brain relaxometry within a rapid 5.5-minute scan time. △ Less

Submitted 22 March, 2024; originally announced March 2024.

Time-Resolved Reconstruction of Motion, Force, and Stiffness using Spectro-Dynamic MRI

Authors: Max H. C. van Riel, Tristan van Leeuwen, Cornelis A. T. van den Berg, Alessandro Sbrizzi

Abstract: Measuring the dynamics and mechanical properties of muscles and joints is important to understand the (patho)physiology of muscles. However, acquiring dynamic time-resolved MRI data is challenging. We have previously developed Spectro-Dynamic MRI which allows the characterization of dynamical systems at a high spatial and temporal resolution directly from k-space data. This work presents an extend… ▽ More Measuring the dynamics and mechanical properties of muscles and joints is important to understand the (patho)physiology of muscles. However, acquiring dynamic time-resolved MRI data is challenging. We have previously developed Spectro-Dynamic MRI which allows the characterization of dynamical systems at a high spatial and temporal resolution directly from k-space data. This work presents an extended Spectro-Dynamic MRI framework that reconstructs 1) time-resolved MR images, 2) time-resolved motion fields, 3) dynamical parameters, and 4) an activation force, at a temporal resolution of 11 ms. An iterative algorithm solves a minimization problem containing four terms: a motion model relating the motion to the fully-sampled k-space data, a dynamical model describing the expected type of dynamics, a data consistency term describing the undersampling pattern, and finally a regularization term for the activation force. We acquired MRI data using a dynamic motion phantom programmed to move like an actively driven linear elastic system, from which all dynamic variables could be accurately reconstructed, regardless of the sampling pattern. The proposed method performed better than a two-step approach, where time-resolved images were first reconstructed from the undersampled data without any information about the motion, followed by a motion estimation step. △ Less

Submitted 11 October, 2023; originally announced October 2023.

Comments: 11 pages, 7 figures, 5 supplementary figures, 1 supplementary video. The video can be viewed by downloading the source file under "Other Formats"

Real-time myocardial landmark tracking for MRI-guided cardiac radio-ablation using Gaussian Processes

Authors: Niek R. F. Huttinga, Osman Akdag, Martin F. Fast, Joost Verhoeff, Firdaus A. A. Mohamed Hoesein, Cornelis A. T. van den Berg, Alessandro Sbrizzi, Stefano Mandija

Abstract: The high speed of cardiorespiratory motion introduces a unique challenge for cardiac stereotactic radio-ablation (STAR) treatments with the MR-linac. Such treatments require tracking myocardial landmarks with a maximum latency of 100 ms, which includes the acquisition of the required data. The aim of this study is to present a new method that allows to track myocardial landmarks from few readouts… ▽ More The high speed of cardiorespiratory motion introduces a unique challenge for cardiac stereotactic radio-ablation (STAR) treatments with the MR-linac. Such treatments require tracking myocardial landmarks with a maximum latency of 100 ms, which includes the acquisition of the required data. The aim of this study is to present a new method that allows to track myocardial landmarks from few readouts of MRI data, thereby achieving a latency sufficient for STAR treatments. We present a tracking framework that requires only few readouts of k-space data as input, which can be acquired at least an order of magnitude faster than MR-images. Combined with the real-time tracking speed of a probabilistic machine learning framework called Gaussian Processes, this allows to track myocardial landmarks with a sufficiently low latency for cardiac STAR guidance, including both the acquisition of required data, and the tracking inference. The framework is demonstrated in 2D on a motion phantom, and in vivo on volunteers and a ventricular tachycardia (arrhythmia) patient. Moreover, the feasibility of an extension to 3D was demonstrated by in silico 3D experiments with a digital motion phantom. The framework was compared with template matching - a reference, image-based, method - and linear regression methods. Results indicate an order of magnitude lower total latency (<10 ms) for the proposed framework in comparison with alternative methods. The root-mean-square-distances and mean end-point-distance with the reference tracking method was less than 0.8 mm for all experiments, showing excellent (sub-voxel) agreement. The high accuracy in combination with a total latency of less than 10 ms - including data acquisition and processing - make the proposed method a suitable candidate for tracking during STAR treatments. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Extending GCC-PHAT using Shift Equivariant Neural Networks

Authors: Axel Berg, Mark O'Connor, Kalle Åström, Magnus Oskarsson

Abstract: Speaker localization using microphone arrays depends on accurate time delay estimation techniques. For decades, methods based on the generalized cross correlation with phase transform (GCC-PHAT) have been widely adopted for this purpose. Recently, the GCC-PHAT has also been used to provide input features to neural networks in order to remove the effects of noise and reverberation, but at the cost… ▽ More Speaker localization using microphone arrays depends on accurate time delay estimation techniques. For decades, methods based on the generalized cross correlation with phase transform (GCC-PHAT) have been widely adopted for this purpose. Recently, the GCC-PHAT has also been used to provide input features to neural networks in order to remove the effects of noise and reverberation, but at the cost of losing theoretical guarantees in noise-free conditions. We propose a novel approach to extending the GCC-PHAT, where the received signals are filtered using a shift equivariant neural network that preserves the timing information contained in the signals. By extensive experiments we show that our model consistently reduces the error of the GCC-PHAT in adverse environments, with guarantees of exact time delay recovery in ideal conditions. △ Less

Submitted 9 August, 2022; originally announced August 2022.

Comments: Proceedings of INTERSPEECH

Journal ref: Proc. Interspeech 2022, 1791-1795

Acceleration Strategies for MR-STAT: Achieving High-Resolution Reconstructions on a Desktop PC within 3 minutes

Authors: Hongyan Liu, Oscar van der Heide, Stefano Mandija, Cornelis A. T. van den Berg, Alessandro Sbrizzi

Abstract: MR-STAT is an emerging quantitative magnetic resonance imaging technique which aims at obtaining multi-parametric tissue parameter maps from single short scans. It describes the relationship between the spatial-domain tissue parameters and the time-domain measured signal by using a comprehensive, volumetric forward model. The MR-STAT reconstruction solves a large-scale nonlinear problem, thus is v… ▽ More MR-STAT is an emerging quantitative magnetic resonance imaging technique which aims at obtaining multi-parametric tissue parameter maps from single short scans. It describes the relationship between the spatial-domain tissue parameters and the time-domain measured signal by using a comprehensive, volumetric forward model. The MR-STAT reconstruction solves a large-scale nonlinear problem, thus is very computationally challenging. In previous work, MR-STAT reconstruction using Cartesian readout data was accelerated by approximating the Hessian matrix with sparse, banded blocks, and can be done on high performance CPU clusters with tens of minutes. In the current work, we propose an accelerated Cartesian MR-STAT algorithm incorporating two different strategies: firstly, a neural network is trained as a fast surrogate to learn the magnetization signal not only in the full time-domain but also in the compressed lowrank domain; secondly, based on the surrogate model, the Cartesian MR-STAT problem is re-formulated and split into smaller sub-problems by the alternating direction method of multipliers. The proposed method substantially reduces the computational requirements for runtime and memory. Simulated and in-vivo balanced MR-STAT experiments show similar reconstruction results using the proposed algorithm compared to the previous sparse Hessian method, and the reconstruction times are at least 40 times shorter. Incorporating sensitivity encoding and regularization terms is straightforward, and allows for better image quality with a negligible increase in reconstruction time. The proposed algorithm could reconstruct both balanced and gradient-spoiled in-vivo data within 3 minutes on a desktop PC, and could thereby facilitate the translation of MR-STAT in clinical settings. △ Less

Submitted 4 May, 2022; originally announced May 2022.

Comments: 12 pages, 7 figures, accepted by IEEE Transactions on Medical Imaging (in press)

Free-breathing motion compensated 4D (3D+respiration) T2-weighted turbo spin-echo MRI for body imaging

Authors: T. Bruijnen, T. Schake, O. Akdag, C. V. M. Bruel, J. J. W. Lagendijk, C. A. T. van den Berg, R. H. N. Tijssen

Abstract: Purpose: To develop and evaluate a free-breathing respiratory motion compensated 4D (3D+respiration) $T_2$-weighted turbo spin echo sequence with application to radiology and MR-guided radiotherapy. Methods: k-space data are continuously acquired using a rewound Cartesian acquisition with spiral profile ordering (rCASPR) to provide matching contrast to the conventional linear phase encode orderi… ▽ More Purpose: To develop and evaluate a free-breathing respiratory motion compensated 4D (3D+respiration) $T_2$-weighted turbo spin echo sequence with application to radiology and MR-guided radiotherapy. Methods: k-space data are continuously acquired using a rewound Cartesian acquisition with spiral profile ordering (rCASPR) to provide matching contrast to the conventional linear phase encode ordering and to sort data into multiple respiratory phases. Low-resolution respiratory-correlated 4D images were reconstructed with compressed sensing and used to estimate non-rigid deformation vector fields, which were subsequently used for a motion compensated image reconstruction. rCASPR sampling was compared to linear and CASPR sampling in terms of point-spread-function (PSF) and image contrast with in silico, phantom and in vivo experiments. Reconstruction parameters for low-resolution 4D-MRI (spatial resolution and temporal regularization) were determined using a grid search. The proposed motion compensated rCASPR was evaluated in eight healthy volunteers and compared to free-breathing scans with linear sampling. Image quality was compared based on visual inspection and quantitatively by means of the gradient entropy. Results: rCASPR provided a superior PSF (similar in ky and narrower in kz) and showed no considerable differences in images contrast compared to linear sampling. The optimal 4D-MRI reconstruction parameters were spatial resolution=$4.5 mm^3$ and $λ_t=10^{-4}$. The groupwise average gradient entropy was 22.31 for linear, 22.20 for rCASPR, 22.14 for soft-gated rCASPR and 22.02 for motion compensated rCASPR. Conclusion: The proposed motion compensated rCASPR enables high quality free-breathing T2-TSE with minimal changes in image contrast and scan time. The proposed method therefore enables direct transfer of clinically used 3D TSE sequences to free-breathing. △ Less

Submitted 7 February, 2022; originally announced February 2022.

Comments: 19 pages, 11 figures

Multi-task fusion for improving mammography screening data classification

Authors: Maria Wimmer, Gert Sluiter, David Major, Dimitrios Lenis, Astrid Berg, Theresa Neubauer, Katja Bühler

Abstract: Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all t… ▽ More Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. We fuse model predictions and high-level features from deep learning models with hybrid patient models to build stronger predictors on patient level. To this end, we propose a multi-branch deep learning model which efficiently fuses features across different tasks and mammograms to obtain a comprehensive patient-level prediction. We train and evaluate our full pipeline on public mammography data, i.e., DDSM and its curated version CBIS-DDSM, and report an AUC score of 0.962 for predicting the presence of any lesion and 0.791 for predicting the presence of malignant lesions on patient level. Overall, our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling. Moreover, by providing not only global patient-level predictions but also task-specific model results that are related to radiological features, our pipeline aims to closely support the reading workflow of radiologists. △ Less

Submitted 1 December, 2021; originally announced December 2021.

Comments: Accepted for publication in IEEE Transactions on Medical Imaging

Real-time non-rigid 3D respiratory motion estimation for MR-guided radiotherapy using MR-MOTUS

Authors: Niek R. F. Huttinga, Tom Bruijnen, Cornelis A. T. van den Berg, Alessandro Sbrizzi

Abstract: The MR-Linac is a combination of an MR-scanner and radiotherapy linear accelerator (Linac) which holds the promise to increase the precision of radiotherapy treatments with MR-guided radiotherapy by monitoring motion during radiotherapy with MRI, and adjusting the radiotherapy plan accordingly. Optimal MR-guidance for respiratory motion during radiotherapy requires MR-based 3D motion estimation wi… ▽ More The MR-Linac is a combination of an MR-scanner and radiotherapy linear accelerator (Linac) which holds the promise to increase the precision of radiotherapy treatments with MR-guided radiotherapy by monitoring motion during radiotherapy with MRI, and adjusting the radiotherapy plan accordingly. Optimal MR-guidance for respiratory motion during radiotherapy requires MR-based 3D motion estimation with a latency of 200-500 ms. Currently this is still challenging since typical methods rely on MR-images, and are therefore limited by the 3D MR-imaging latency. In this work, we present a method to perform non-rigid 3D respiratory motion estimation with 170 ms latency, including both acquisition and reconstruction. The proposed method called real-time low-rank MR-MOTUS reconstructs motion-fields directly from k-space data, and leverages an explicit low-rank decomposition of motion-fields to split the large scale 3D+t motion-field reconstruction problem posed in our previous work into two parts: (I) a medium-scale offline preparation phase and (II) a small-scale online inference phase which exploits the results of the offline phase for real-time computations. The method was validated on free-breathing data of five volunteers, acquired with a 1.5T Elekta Unity MR-Linac. Results show that the reconstructed 3D motion-field are anatomically plausible, highly correlated with a self-navigation motion surrogate (R = 0.975 +/- 0.0110), and can be reconstructed with a total latency of 170 ms that is sufficient for real-time MR-guided abdominal radiotherapy. △ Less

Submitted 14 September, 2021; v1 submitted 16 April, 2021; originally announced April 2021.

Comments: This manuscript has supplementary files which can be downloaded at https://surfdrive.surf.nl/files/index.php/s/vz2xmwliglRmcjo. The files include six videos that show reconstructed motion-fields and a document with supporting figures. See Appendix I for a description of all individual files

Keyword Transformer: A Self-Attention Model for Keyword Spotting

Authors: Axel Berg, Mark O'Connor, Miguel Tairum Cruz

Abstract: The Transformer architecture has been successful across many domains, including natural language processing, computer vision and speech recognition. In keyword spotting, self-attention has primarily been used on top of convolutional or recurrent encoders. We investigate a range of ways to adapt the Transformer architecture to keyword spotting and introduce the Keyword Transformer (KWT), a fully se… ▽ More The Transformer architecture has been successful across many domains, including natural language processing, computer vision and speech recognition. In keyword spotting, self-attention has primarily been used on top of convolutional or recurrent encoders. We investigate a range of ways to adapt the Transformer architecture to keyword spotting and introduce the Keyword Transformer (KWT), a fully self-attentional architecture that exceeds state-of-the-art performance across multiple tasks without any pre-training or additional data. Surprisingly, this simple architecture outperforms more complex models that mix convolutional, recurrent and attentive layers. KWT can be used as a drop-in replacement for these models, setting two new benchmark records on the Google Speech Commands dataset with 98.6% and 97.7% accuracy on the 12 and 35-command tasks respectively. △ Less

Submitted 15 June, 2021; v1 submitted 1 April, 2021; originally announced April 2021.

Comments: Proceedings of INTERSPEECH

Journal ref: Proc. Interspeech 2021, 4249-4253

Soft Tissue Sarcoma Co-Segmentation in Combined MRI and PET/CT Data

Authors: Theresa Neubauer, Maria Wimmer, Astrid Berg, David Major, Dimitrios Lenis, Thomas Beyer, Jelena Saponjski, Katja Bühler

Abstract: Tumor segmentation in multimodal medical images has seen a growing trend towards deep learning based methods. Typically, studies dealing with this topic fuse multimodal image data to improve the tumor segmentation contour for a single imaging modality. However, they do not take into account that tumor characteristics are emphasized differently by each modality, which affects the tumor delineation.… ▽ More Tumor segmentation in multimodal medical images has seen a growing trend towards deep learning based methods. Typically, studies dealing with this topic fuse multimodal image data to improve the tumor segmentation contour for a single imaging modality. However, they do not take into account that tumor characteristics are emphasized differently by each modality, which affects the tumor delineation. Thus, the tumor segmentation is modality- and task-dependent. This is especially the case for soft tissue sarcomas, where, due to necrotic tumor tissue, the segmentation differs vastly. Closing this gap, we develop a modalityspecific sarcoma segmentation model that utilizes multimodal image data to improve the tumor delineation on each individual modality. We propose a simultaneous co-segmentation method, which enables multimodal feature learning through modality-specific encoder and decoder branches, and the use of resource-effcient densely connected convolutional layers. We further conduct experiments to analyze how different input modalities and encoder-decoder fusion strategies affect the segmentation result. We demonstrate the effectiveness of our approach on public soft tissue sarcoma data, which comprises MRI (T1 and T2 sequence) and PET/CT scans. The results show that our multimodal co-segmentation model provides better modality-specific tumor segmentation than models using only the PET or MRI (T1 and T2) scan as input. △ Less

Submitted 24 September, 2020; v1 submitted 28 August, 2020; originally announced August 2020.

Comments: Accepted for publication at Multimodal Learning for Clinical Decision Support Workshop at MICCAI 2020 (edit: corrected typos and model name in Fig. 3, added missing circles in Table 1)

Fast and Accurate Modeling of Transient-State Gradient-Spoiled Sequences by Recurrent Neural Networks

Authors: Hongyan Liu, Oscar van der Heide, Cornelis A. T. van den Berg, Alessandro Sbrizzi

Abstract: Fast and accurate modeling of MR signal responses are typically required for various quantitative MRI applications, such as MR Fingerprinting and MR-STAT. This work uses a new EPG-Bloch model for accurate simulation of transient-state gradient-spoiled MR sequences, and proposes a Recurrent Neural Network (RNN) as a fast surrogate of the EPG-Bloch model for computing large-scale MR signals and deri… ▽ More Fast and accurate modeling of MR signal responses are typically required for various quantitative MRI applications, such as MR Fingerprinting and MR-STAT. This work uses a new EPG-Bloch model for accurate simulation of transient-state gradient-spoiled MR sequences, and proposes a Recurrent Neural Network (RNN) as a fast surrogate of the EPG-Bloch model for computing large-scale MR signals and derivatives. The computational efficiency of the RNN model is demonstrated by comparing with other existing models, showing one to three orders of acceleration comparing to the latest GPU-accelerated open-source EPG package. By using numerical and in-vivo brain data, two use cases, namely MRF dictionary generation and optimal experimental design, are also provided. Results show that the RNN surrogate model can be efficiently used for computing large-scale dictionaries of transient-states signals and derivatives within tens of seconds, resulting in several orders of magnitude acceleration with respect to state-of-the-art implementations. The practical application of transient-states quantitative techniques can therefore be substantially facilitated. △ Less

Submitted 21 August, 2020; v1 submitted 17 August, 2020; originally announced August 2020.

Comments: Correct for typo errors

Non-rigid 3D motion estimation at high temporal resolution from prospectively undersampled k-space data using low-rank MR-MOTUS

Authors: Niek R. F. Huttinga, Tom Bruijnen, Cornelis A. T. van den Berg, Alessandro Sbrizzi

Abstract: With the recent introduction of the MR-LINAC, an MR-scanner combined with a radiotherapy LINAC, MR-based motion estimation has become of increasing interest to (retrospectively) characterize tumor and organs-at-risk motion during radiotherapy. To this extent, we introduce low-rank MR-MOTUS, a framework to retrospectively reconstruct time-resolved non-rigid 3D+t motion-fields from a single low-reso… ▽ More With the recent introduction of the MR-LINAC, an MR-scanner combined with a radiotherapy LINAC, MR-based motion estimation has become of increasing interest to (retrospectively) characterize tumor and organs-at-risk motion during radiotherapy. To this extent, we introduce low-rank MR-MOTUS, a framework to retrospectively reconstruct time-resolved non-rigid 3D+t motion-fields from a single low-resolution reference image and prospectively undersampled k-space data acquired during motion. Low-rank MR-MOTUS exploits spatio-temporal correlations in internal body motion with a low-rank motion model, and inverts a signal model that relates motion-fields directly to a reference image and k-space data. The low-rank model reduces the degrees-of-freedom, memory consumption and reconstruction times by assuming a factorization of space-time motion-fields in spatial and temporal components. Low-rank MR-MOTUS was employed to estimate motion in 2D/3D abdominothoracic scans and 3D head scans. Data were acquired using golden-ratio radial readouts. Reconstructed 2D and 3D respiratory motion-fields were respectively validated against time-resolved and respiratory-resolved image reconstructions, and the head motion against static image reconstructions from fully-sampled data acquired right before and right after the motion. Results show that 2D+t respiratory motion can be estimated retrospectively at 40.8 motion-fields-per-second, 3D+t respiratory motion at 7.6 motion-fields-per-second and 3D+t head-neck motion at 9.3 motion-fields-per-second. The validations show good consistency with image reconstructions. The proposed framework can estimate time-resolved non-rigid 3D motion-fields, which allows to characterize drifts and intra and inter-cycle patterns in breathing motion during radiotherapy, and could form the basis for real-time MR-guided radiotherapy. △ Less

Submitted 1 July, 2020; originally announced July 2020.

Comments: 18 pages main text, 8 main figures, 1 main table, 12 supporting videos, 2 supporting figures, 1 supporting information PDF. Submitted to Magnetic Resonance in Medicine as Full Paper

LU-Net: a multi-task network to improve the robustness of segmentation of left ventriclular structures by deep learning in 2D echocardiography

Authors: Sarah Leclerc, Erik Smistad, Andreas Østvik, Frederic Cervenansky, Florian Espinosa, Torvald Espeland, Erik Andreas Rye Berg, Thomas Grenier, Carole Lartizien, Pierre-Marc Jodoin, Lasse Lovstakken, Olivier Bernard

Abstract: Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semi-automatically in clinical routine, and is thus prone to inter- and intra-observer variability. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a… ▽ More Segmentation of cardiac structures is one of the fundamental steps to estimate volumetric indices of the heart. This step is still performed semi-automatically in clinical routine, and is thus prone to inter- and intra-observer variability. Recent studies have shown that deep learning has the potential to perform fully automatic segmentation. However, the current best solutions still suffer from a lack of robustness. In this work, we introduce an end-to-end multi-task network designed to improve the overall accuracy of cardiac segmentation while enhancing the estimation of clinical indices and reducing the number of outliers. Results obtained on a large open access dataset show that our method outperforms the current best performing deep learning solution and achieved an overall segmentation accuracy lower than the intra-observer variability for the epicardial border (i.e. on average a mean absolute error of 1.5mm and a Hausdorff distance of 5.1mm) with 11% of outliers. Moreover, we demonstrate that our method can closely reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.96 and a mean absolute error of 7.6ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.83 and an absolute mean error of 5.0%, producing scores that are slightly below the intra-observer margin. Based on this observation, areas for improvement are suggested. △ Less

Submitted 4 April, 2020; originally announced April 2020.

Interpreting Medical Image Classifiers by Optimization Based Counterfactual Impact Analysis

Authors: David Major, Dimitrios Lenis, Maria Wimmer, Gert Sluiter, Astrid Berg, Katja Bühler

Abstract: Clinical applicability of automated decision support systems depends on a robust, well-understood classification interpretation. Artificial neural networks while achieving class-leading scores fall short in this regard. Therefore, numerous approaches have been proposed that map a salient region of an image to a diagnostic classification. Utilizing heuristic methodology, like blurring and noise, th… ▽ More Clinical applicability of automated decision support systems depends on a robust, well-understood classification interpretation. Artificial neural networks while achieving class-leading scores fall short in this regard. Therefore, numerous approaches have been proposed that map a salient region of an image to a diagnostic classification. Utilizing heuristic methodology, like blurring and noise, they tend to produce diffuse, sometimes misleading results, hindering their general adoption. In this work we overcome these issues by presenting a model agnostic saliency map** framework tailored to medical imaging. We replace heuristic techniques with a strong neighborhood conditioned inpainting approach, which avoids anatomically implausible artefacts. We formulate saliency attribution as a map-quality optimization task, enforcing constrained and focused attributions. Experiments on public mammography data show quantitatively and qualitatively more precise localization and clearer conveying results than existing state-of-the-art methods. △ Less

Submitted 3 April, 2020; originally announced April 2020.

Comments: Accepted for publication at IEEE International Symposium on Biomedical Imaging (ISBI) 2020

CBCT-to-CT synthesis with a single neural network for head-and-neck, lung and breast cancer adaptive radiotherapy

Authors: Matteo Maspero, Mark HF Savenije, Tristan CF van Heijst, Joost JC Verhoeff, Alexis NTJ Kotte, Anette C Houweling, Cornelis AT van den Berg

Abstract: Purpose: CBCT-based adaptive radiotherapy requires daily images for accurate dose calculations. This study investigates the feasibility of applying a single convolutional network to facilitate CBCT-to-CT synthesis for head-and-neck, lung, and breast cancer patients. Methods: Ninety-nine patients diagnosed with head-and-neck, lung or breast cancer undergoing radiotherapy with CBCT-based position ve… ▽ More Purpose: CBCT-based adaptive radiotherapy requires daily images for accurate dose calculations. This study investigates the feasibility of applying a single convolutional network to facilitate CBCT-to-CT synthesis for head-and-neck, lung, and breast cancer patients. Methods: Ninety-nine patients diagnosed with head-and-neck, lung or breast cancer undergoing radiotherapy with CBCT-based position verification were included in this study. CBCTs were registered to planning CTs according to clinical procedures. Three cycle-consistent generative adversarial networks (cycle-GANs) were trained in an unpaired manner on 15 patients per anatomical site generating synthetic-CTs (sCTs). Another network was trained with all the anatomical sites together. Performances of all four networks were compared and evaluated for image similarity against rescan CT (rCT). Clinical plans were recalculated on CT and sCT and analysed through voxel-based dose differences and γ-analysis. Results: A sCT was generated in 10 seconds. Image similarity was comparable between models trained on different anatomical sites and a single model for all sites. Mean dose differences < 0.5% were obtained in high-dose regions. Mean gamma (2%,2mm) pass-rates > 95% were achieved for all sites. Conclusions: Cycle-GAN reduced CBCT artefacts and increased HU similarity to CT, enabling sCT-based dose calculations. The speed of the network can facilitate on-line adaptive radiotherapy using a single network for head-and-neck, lung and breast cancer patients. △ Less

Submitted 23 December, 2019; originally announced December 2019.

Comments: Submitted to Medical Physics; 2019-12-23

Journal ref: Physics and Imaging in Radiation Oncology Volume 14, April 2020, Pages 24-31

Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography

Authors: Sarah Leclerc, Erik Smistad, João Pedrosa, Andreas Østvik, Frederic Cervenansky, Florian Espinosa, Torvald Espeland, Erik Andreas Rye Berg, Pierre-Marc Jodoin, Thomas Grenier, Carole Lartizien, Jan D'hooge, Lasse Lovstakken, Olivier Bernard

Abstract: Delineation of the cardiac structures from 2D echocardiographic images is a common clinical task to establish a diagnosis. Over the past decades, the automation of this task has been the subject of intense research. In this paper, we evaluate how far the state-of-the-art encoder-decoder deep convolutional neural network methods can go at assessing 2D echocardiographic images, i.e segmenting cardia… ▽ More Delineation of the cardiac structures from 2D echocardiographic images is a common clinical task to establish a diagnosis. Over the past decades, the automation of this task has been the subject of intense research. In this paper, we evaluate how far the state-of-the-art encoder-decoder deep convolutional neural network methods can go at assessing 2D echocardiographic images, i.e segmenting cardiac structures as well as estimating clinical indices, on a dataset especially designed to answer this objective. We therefore introduce the Cardiac Acquisitions for Multi-structure Ultrasound Segmentation (CAMUS) dataset, the largest publicly-available and fully-annotated dataset for the purpose of echocardiographic assessment. The dataset contains two and four-chamber acquisitions from 500 patients with reference measurements from one cardiologist on the full dataset and from three cardiologists on a fold of 50 patients. Results show that encoder-decoder based architectures outperform state-of-the-art non-deep learning methods and faithfully reproduce the expert analysis for the end-diastolic and end-systolic left ventricular volumes, with a mean correlation of 0.95 and an absolute mean error of 9.5 ml. Concerning the ejection fraction of the left ventricle, results are more contrasted with a mean correlation coefficient of 0.80 and an absolute mean error of 5.6 %. Although these results are below the inter-observer scores, they remain slightly worse than the intra-observer's ones. Based on this observation, areas for improvement are defined, which open the door for accurate and fully-automatic analysis of 2D echocardiographic images. △ Less

Submitted 22 August, 2019; v1 submitted 16 August, 2019; originally announced August 2019.

Combining Deep Learning and 3D Contrast Source Inversion in MR-based Electrical Properties Tomography

Authors: Reijer L. Leijsen, Cornelis A. T. van den Berg, Andrew G. Webb, Rob F. Remis, Stefano Mandija

Abstract: Magnetic resonance-electrical properties tomography (MR-EPT) is a technique used to estimate the conductivity and permittivity of tissues from MR measurements of the transmit magnetic field. Different reconstruction methods are available, however all these methods present several limitations which hamper the clinical applicability. Standard Helmholtz based MR-EPT methods are severely affected by n… ▽ More Magnetic resonance-electrical properties tomography (MR-EPT) is a technique used to estimate the conductivity and permittivity of tissues from MR measurements of the transmit magnetic field. Different reconstruction methods are available, however all these methods present several limitations which hamper the clinical applicability. Standard Helmholtz based MR-EPT methods are severely affected by noise. Iterative reconstruction methods such as contrast source inversion-EPT (CSI-EPT) are typically time consuming and are dependent on their initialization. Deep learning (DL) based methods require a large amount of training data before sufficient generalization can be achieved. Here, we investigate the benefits achievable using a hybrid approach, i.e. using MR-EPT or DL-EPT as initialization guesses for standard 3D CSI-EPT. Using realistic electromagnetic simulations at 3 T and 7 T, the accuracy and precision of hybrid CSI reconstructions are compared to standard 3D CSI-EPT reconstructions. Our results indicate that a hybrid method consisting of an initial DL-EPT reconstruction followed by a 3D CSI-EPT reconstruction would be beneficial. DL-EPT combined with standard 3D CSI-EPT exploits the power of data driven DL-based EPT reconstructions while the subsequent CSI-EPT facilitates a better generalization by providing data consistency. △ Less

Submitted 13 August, 2019; originally announced August 2019.

Comments: 8 pages, 4 figures, 1 table

Deep learning brain conductivity map** using a patch-based 3D U-net

Authors: Nils Hampe, Ulrich Katscher, Cornelis A. T. van den Berg, Khin Khin Tha, Stefano Mandija

Abstract: Purpose: To investigate deep learning electrical properties tomography (EPT) for application on different simulated and in-vivo datasets including pathologies for obtaining quantitative brain conductivity maps. Methods: 3D patch-based convolutional neural networks were trained to predict conductivity maps from B1 transceive phase data. To compare the performance of DLEPT networks on different data… ▽ More Purpose: To investigate deep learning electrical properties tomography (EPT) for application on different simulated and in-vivo datasets including pathologies for obtaining quantitative brain conductivity maps. Methods: 3D patch-based convolutional neural networks were trained to predict conductivity maps from B1 transceive phase data. To compare the performance of DLEPT networks on different datasets, three datasets were used throughout this work, one from simulations and two from in-vivo measurements from healthy volunteers and cancer patients, respectively. At first, networks trained on simulations are tested on all datasets with different levels of homogeneous Gaussian noise introduced in training and testing. Secondly, to investigate potential robustness towards systematical differences between simulated and measured phase maps, in-vivo data with conductivity labels from conventional EPT is used for training. Results: High quality of reconstructions from networks trained on simulations with and without noise confirms the potential of deep learning for EPT. However, artifact encumbered results in this work uncover challenges in application of DLEPT to in-vivo data. Training DLEPT networks on conductivity labels from conventional EPT improves quality of results. This is argued to be caused by robustness to artifacts from image acquisition. Conclusions: Networks trained on simulations with added homogeneous Gaussian noise yield reconstruction artifacts when applied to in-vivo data. Training with realistic phase data and conductivity labels from conventional EPT allows for severely reducing these artifacts. △ Less

Submitted 12 August, 2019; originally announced August 2019.

Deep Learning Segmentation in 2D echocardiography using the CAMUS dataset : Automatic Assessment of the Anatomical Shape Validity

Authors: Sarah Leclerc, Erik Smistad, Andreas Østvik, Frederic Cervenansky, Florian Espinosa, Torvald Espeland, Erik Andreas Rye Berg, Pierre-Marc Jodoin, Thomas Grenier, Carole Lartizien, Lasse Lovstakken, Olivier Bernard

Abstract: We recently published a deep learning study on the potential of encoder-decoder networks for the segmentation of the 2D CAMUS ultrasound dataset. We propose in this abstract an extension of the evaluation criteria to anatomical assessment, as traditional geometric and clinical metrics in cardiac segmentation do not take into account the anatomical correctness of the predicted shapes. The completed… ▽ More We recently published a deep learning study on the potential of encoder-decoder networks for the segmentation of the 2D CAMUS ultrasound dataset. We propose in this abstract an extension of the evaluation criteria to anatomical assessment, as traditional geometric and clinical metrics in cardiac segmentation do not take into account the anatomical correctness of the predicted shapes. The completed study sheds a new light on the ranking of models. △ Less

Submitted 8 August, 2019; originally announced August 2019.

Comments: MIDL 2019 [arXiv:1907.08612]

Report number: MIDL/2019/ExtendedAbstract/Byx4AM1ntN

Unsupervised Learning of Anomaly Detection from Contaminated Image Data using Simultaneous Encoder Training

Authors: Amanda Berg, Jörgen Ahlberg, Michael Felsberg

Abstract: Unsupervised learning of anomaly detection in high-dimensional data, such as images, is a challenging problem recently subject to intense research. Through careful modelling of the data distribution of normal samples, it is possible to detect deviant samples, so called anomalies. Generative Adversarial Networks (GANs) can model the highly complex, high-dimensional data distribution of normal image… ▽ More Unsupervised learning of anomaly detection in high-dimensional data, such as images, is a challenging problem recently subject to intense research. Through careful modelling of the data distribution of normal samples, it is possible to detect deviant samples, so called anomalies. Generative Adversarial Networks (GANs) can model the highly complex, high-dimensional data distribution of normal image samples, and have shown to be a suitable approach to the problem. Previously published GAN-based anomaly detection methods often assume that anomaly-free data is available for training. However, this assumption is not valid in most real-life scenarios, a.k.a. in the wild. In this work, we evaluate the effects of anomaly contaminations in the training data on state-of-the-art GAN-based anomaly detection methods. As expected, detection performance deteriorates. To address this performance drop, we propose to add an additional encoder network already at training time and show that joint generator-encoder training stratifies the latent space, mitigating the problem with contaminated data. We show experimentally that the norm of a query image in this stratified latent space becomes a highly significant cue to discriminate anomalies from normal data. The proposed method achieves state-of-the-art performance on CIFAR-10 as well as on a large, previously untested dataset with cell images. △ Less

Submitted 20 November, 2019; v1 submitted 27 May, 2019; originally announced May 2019.

Opening a new window on MR-based Electrical Properties Tomography with deep learning

Authors: Stefano Mandija, Ettore F. Meliadò, Niek R. F. Huttinga, Peter R. Luijten, Cornelis A. T. van den Berg

Abstract: Electrical properties (EPs) of tissues, conductivity and permittivity, are modulated by the ionic and water content, which change in presence of pathologies. Information on tissues EPs can be used e.g. as an endogenous biomarker in oncology. MR-Electrical Properties Tomography (MR-EPT) aims to reconstruct tissue EPs by solving an electromagnetic inverse problem relating MR measurements of the tran… ▽ More Electrical properties (EPs) of tissues, conductivity and permittivity, are modulated by the ionic and water content, which change in presence of pathologies. Information on tissues EPs can be used e.g. as an endogenous biomarker in oncology. MR-Electrical Properties Tomography (MR-EPT) aims to reconstruct tissue EPs by solving an electromagnetic inverse problem relating MR measurements of the transmit radiofrequency RF field to the EPs. However, MR-EPT reconstructions highly suffer from noise in the RF field maps, which limits the clinical applicability. Instead of employing electromagnetic models posing strict requirements on the measured quantities, we propose a data driven approach where the inverse transformation is learned by means of a neural network. Supervised training of a conditional generative adversarial neural network was performed using simulated realistic RF field maps and realistic human head dielectric models. Deep learning EPT (DL-EPT) reconstructions are presented for in-silica MR data and MR measurements at 3 Tesla on phantoms and human brains. DL-EPT shows high quality EP maps, demonstrating good accuracy and greatly improved precision compared to conventional MR-EPT. Moreover, DL-EPT allows permittivity reconstructions at 3 Tesla, which is not possible with state-of-art MR-EPT techniques. The supervised learning-based approach leverages the strength of tailored electromagnetic simulations, allowing inclusion of a priori information (e.g. coil setup) and circumvention of inaccessible MR electromagnetic quantities. Since DL-EPT is highly noise-robust, the requirements for MRI data acquisitions can be relaxed, allowing faster acquisitions and higher resolutions. We believe that DL-EPT greatly improves the quality and applicability of EPT opening a new window for an endogenous biomarker in MRI diagnostics that reflects differences in ionic tissue content. △ Less

Submitted 19 August, 2019; v1 submitted 30 March, 2018; originally announced April 2018.

Journal ref: Published online in: Scientific Reports (Jun-2019) 9: 8895

SRE: Semantic Rules Engine For the Industrial Internet-Of-Things Gateways

Authors: Charbel El Kaed, Imran Khan, Andre Van Den Berg, Hicham Hossayni, Christophe Saint-Marcel

Abstract: The Advent of the Internet-of-Things (IoT) paradigm has brought opportunities to solve many real-world problems. Energy management, for example, has attracted huge interest from academia, industries, governments and regulatory bodies. It involves collecting energy usage data, analyzing it, and optimizing the energy consumption by applying control strategies. However, in industrial environments, pe… ▽ More The Advent of the Internet-of-Things (IoT) paradigm has brought opportunities to solve many real-world problems. Energy management, for example, has attracted huge interest from academia, industries, governments and regulatory bodies. It involves collecting energy usage data, analyzing it, and optimizing the energy consumption by applying control strategies. However, in industrial environments, performing such optimization is not trivial. The changes in business rules, process control, and customer requirements make it much more challenging. In this paper, a Semantic Rules Engine (SRE) for industrial gateways is presented that allows implementing dynamic and flexible rule-based control strategies. It is simple, expressive, and allows managing rules on-the-fly without causing any service interruption. Additionally, it can handle semantic queries and provide results by inferring additional knowledge from previously defined concepts in ontologies. SRE has been validated and tested on different hardware platforms and in commercial products. Performance evaluations are also presented to validate its conformance to the customer requirements. △ Less

Submitted 26 October, 2017; originally announced October 2017.

Comments: Accepted for publication in forthcoming issue of IEEE Transactions on Industrial Informatics. The content is final but has NOT been proof-read

Journal ref: IEEE Transactions on Industrial Informatics, 2017