Showing 1–32 of 32 results for author: Viergever, M A

Effect of latent space distribution on the segmentation of images with multiple annotations

Authors: Ishaan Bhat, Josien P. W. Pluim, Max A. Viergever, Hugo J. Kuijf

Abstract: We propose the Generalized Probabilistic U-Net, which extends the Probabilistic U-Net by allowing more general forms of the Gaussian distribution as the latent space distribution that can better approximate the uncertainty in the reference segmentations. We study the effect the choice of latent space distribution has on capturing the variation in the reference segmentations for lung tumors and whi… ▽ More We propose the Generalized Probabilistic U-Net, which extends the Probabilistic U-Net by allowing more general forms of the Gaussian distribution as the latent space distribution that can better approximate the uncertainty in the reference segmentations. We study the effect the choice of latent space distribution has on capturing the variation in the reference segmentations for lung tumors and white matter hyperintensities in the brain. We show that the choice of distribution affects the sample diversity of the predictions and their overlap with respect to the reference segmentations. We have made our implementation available at https://github.com/ishaanb92/GeneralizedProbabilisticUNet △ Less

Submitted 26 April, 2023; originally announced April 2023.

Comments: Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2023:005. arXiv admin note: text overlap with arXiv:2207.12872

Journal ref: Machine.Learning.for.Biomedical.Imaging. 2 (2023)

Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection

Authors: Ishaan Bhat, Josien P. W. Pluim, Max A. Viergever, Hugo J. Kuijf

Abstract: Deep learning techniques show success in detecting objects in medical images, but still suffer from false-positive predictions that may hinder accurate diagnosis. The estimated uncertainty of the neural network output has been used to flag incorrect predictions. We study the role played by features computed from neural network uncertainty estimates and shape-based features computed from binary pre… ▽ More Deep learning techniques show success in detecting objects in medical images, but still suffer from false-positive predictions that may hinder accurate diagnosis. The estimated uncertainty of the neural network output has been used to flag incorrect predictions. We study the role played by features computed from neural network uncertainty estimates and shape-based features computed from binary predictions in reducing false positives in liver lesion detection by develo** a classification-based post-processing step for different uncertainty estimation methods. We demonstrate an improvement in the lesion detection performance of the neural network (with respect to F1-score) for all uncertainty estimation methods on two datasets, comprising abdominal MR and CT images, respectively. We show that features computed from neural network uncertainty estimates tend not to contribute much toward reducing false positives. Our results show that factors like class imbalance (true over false positive ratio) and shape-based features extracted from uncertainty maps play an important role in distinguishing false positive from true positive predictions. Our code can be found at https://github.com/ishaanb92/FPCPipeline. △ Less

Submitted 15 December, 2022; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: Accepted for publication in the Journal of Machine Learning for Biomedical Imaging (MELBA)

Journal ref: https://www.melba-journal.org/papers/2022:030.html

Generative Models for Reproducible Coronary Calcium Scoring

Authors: Sanne G. M. van Velzen, Bob D. de Vos, Julia M. H. Noothout, Helena M. Verkooijen, Max A. Viergever, Ivana Išgum

Abstract: Purpose: Coronary artery calcium (CAC) score, i.e. the amount of CAC quantified in CT, is a strong and independent predictor of coronary heart disease (CHD) events. However, CAC scoring suffers from limited interscan reproducibility, which is mainly due to the clinical definition requiring application of a fixed intensity level threshold for segmentation of calcifications. This limitation is espec… ▽ More Purpose: Coronary artery calcium (CAC) score, i.e. the amount of CAC quantified in CT, is a strong and independent predictor of coronary heart disease (CHD) events. However, CAC scoring suffers from limited interscan reproducibility, which is mainly due to the clinical definition requiring application of a fixed intensity level threshold for segmentation of calcifications. This limitation is especially pronounced in non-ECG-synchronized CT where lesions are more impacted by cardiac motion and partial volume effects. Therefore, we propose a CAC quantification method that does not require a threshold for segmentation of CAC. Approach: Our method utilizes a generative adversarial network where a CT with CAC is decomposed into an image without CAC and an image showing only CAC. The method, using a CycleGAN, was trained using 626 low-dose chest CTs and 514 radiotherapy treatment planning CTs. Interscan reproducibility was compared to clinical calcium scoring in radiotherapy treatment planning CTs of 1,662 patients, each having two scans. Results: A lower relative interscan difference in CAC mass was achieved by the proposed method: 47% compared to 89% manual clinical calcium scoring. The intraclass correlation coefficient of Agatston scores was 0.96 for the proposed method compared to 0.91 for automatic clinical calcium scoring. Conclusions: The increased interscan reproducibility achieved by our method may lead to increased reliability of CHD risk categorization and improved accuracy of CHD event prediction. △ Less

Submitted 24 May, 2022; originally announced May 2022.

Comments: In press

Journal ref: Journal of Medical Imaging 9(5) 052406 (31 May 2022)

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

Authors: Bas H. M. van der Velden, Hugo J. Kuijf, Kenneth G. A. Gilhuijs, Max A. Viergever

Abstract: With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image an… ▽ More With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis. △ Less

Submitted 22 July, 2021; originally announced July 2021.

Comments: Submitted for publication. Comments welcome by email to first author

Journal ref: Medical Image Analysis 2022

Deep Learning from Dual-Energy Information for Whole-Heart Segmentation in Dual-Energy and Single-Energy Non-Contrast-Enhanced Cardiac CT

Authors: Steffen Bruns, Jelmer M. Wolterink, Richard A. P. Takx, Robbert W. van Hamersvelt, Dominika Suchá, Max A. Viergever, Tim Leiner, Ivana Išgum

Abstract: Deep learning-based whole-heart segmentation in coronary CT angiography (CCTA) allows the extraction of quantitative imaging measures for cardiovascular risk prediction. Automatic extraction of these measures in patients undergoing only non-contrast-enhanced CT (NCCT) scanning would be valuable. In this work, we leverage information provided by a dual-layer detector CT scanner to obtain a referenc… ▽ More Deep learning-based whole-heart segmentation in coronary CT angiography (CCTA) allows the extraction of quantitative imaging measures for cardiovascular risk prediction. Automatic extraction of these measures in patients undergoing only non-contrast-enhanced CT (NCCT) scanning would be valuable. In this work, we leverage information provided by a dual-layer detector CT scanner to obtain a reference standard in virtual non-contrast (VNC) CT images mimicking NCCT images, and train a 3D convolutional neural network (CNN) for the segmentation of VNC as well as NCCT images. Contrast-enhanced acquisitions on a dual-layer detector CT scanner were reconstructed into a CCTA and a perfectly aligned VNC image. In each CCTA image, manual reference segmentations of the left ventricular (LV) myocardium, LV cavity, right ventricle, left atrium, right atrium, ascending aorta, and pulmonary artery trunk were obtained and propagated to the corresponding VNC image. These VNC images and reference segmentations were used to train 3D CNNs for automatic segmentation in either VNC images or NCCT images. Automatic segmentations in VNC images showed good agreement with reference segmentations, with an average Dice similarity coefficient of 0.897 \pm 0.034 and an average symmetric surface distance of 1.42 \pm 0.45 mm. Volume differences [95% confidence interval] between automatic NCCT and reference CCTA segmentations were -19 [-67; 30] mL for LV myocardium, -25 [-78; 29] mL for LV cavity, -29 [-73; 14] mL for right ventricle, -20 [-62; 21] mL for left atrium, and -19 [-73; 34] mL for right atrium, respectively. In 214 (74%) NCCT images from an independent multi-vendor multi-center set, two observers agreed that the automatic segmentation was mostly accurate or better. This method might enable quantification of additional cardiac measures from NCCT images for improved cardiovascular risk prediction. △ Less

Submitted 10 August, 2020; originally announced August 2020.

Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images

Authors: Julia M. H. Noothout, Bob D. de Vos, Jelmer M. Wolterink, Elbrich M. Postma, Paul A. M. Smeets, Richard A. P. Takx, Tim Leiner, Max A. Viergever, Ivana Išgum

Abstract: In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vector… ▽ More In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage. △ Less

Submitted 10 July, 2020; originally announced July 2020.

Comments: 12 pages, accepted at IEEE transactions in Medical Imaging

Combined analysis of coronary arteries and the left ventricular myocardium in cardiac CT angiography for detection of patients with functionally significant stenosis

Authors: Majd Zreik, Tim Leiner, Nadieh Khalili, Robbert W. van Hamersvelt, Jelmer M. Wolterink, Michiel Voskuil, Max A. Viergever, Ivana Išgum

Abstract: Treatment of patients with obstructive coronary artery disease is guided by the functional significance of a coronary artery stenosis. Fractional flow reserve (FFR), measured during invasive coronary angiography (ICA), is considered the gold standard to define the functional significance of a coronary stenosis. Here, we present a method for non-invasive detection of patients with functionally sign… ▽ More Treatment of patients with obstructive coronary artery disease is guided by the functional significance of a coronary artery stenosis. Fractional flow reserve (FFR), measured during invasive coronary angiography (ICA), is considered the gold standard to define the functional significance of a coronary stenosis. Here, we present a method for non-invasive detection of patients with functionally significant coronary artery stenosis, combining analysis of the coronary artery tree and the left ventricular (LV) myocardium in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA scans of 126 patients who underwent invasive FFR measurements, to determine the functional significance of coronary stenoses. We combine our previous works for the analysis of the complete coronary artery tree and the LV myocardium: Coronary arteries are encoded by two disjoint convolutional autoencoders (CAEs) and the LV myocardium is characterized by a convolutional neural network (CNN) and a CAE. Thereafter, using the extracted encodings of all coronary arteries and the LV myocardium, patients are classified according to the presence of functionally significant stenosis, as defined by the invasively measured FFR. To handle the varying number of coronary arteries in a patient, the classification is formulated as a multiple instance learning problem and is performed using an attention-based neural network. Cross-validation experiments resulted in an average area under the receiver operating characteristic curve of $0.74 \pm 0.01$, and showed that the proposed combined analysis outperformed the analysis of the coronary arteries or the LV myocardium only. The results demonstrate the feasibility of combining the analyses of the complete coronary artery tree and the LV myocardium in CCTA images for the detection of patients with functionally significant stenosis in coronary arteries. △ Less

Submitted 10 November, 2019; originally announced November 2019.

Comments: Submitted to IEEE Access. arXiv admin note: text overlap with arXiv:1906.04419

Liver segmentation and metastases detection in MR images using convolutional neural networks

Authors: Mariëlle J. A. Jansen, Hugo J. Kuijf, Maarten Niekel, Wouter B. Veldhuis, Frank J. Wessels, Max A. Viergever, Josien P. W. Pluim

Abstract: Primary tumors have a high likelihood of develo** metastases in the liver and early detection of these metastases is crucial for patient outcome. We propose a method based on convolutional neural networks (CNN) to detect liver metastases. First, the liver was automatically segmented using the six phases of abdominal dynamic contrast enhanced (DCE) MR images. Next, DCE-MR and diffusion weighted (… ▽ More Primary tumors have a high likelihood of develo** metastases in the liver and early detection of these metastases is crucial for patient outcome. We propose a method based on convolutional neural networks (CNN) to detect liver metastases. First, the liver was automatically segmented using the six phases of abdominal dynamic contrast enhanced (DCE) MR images. Next, DCE-MR and diffusion weighted (DW) MR images are used for metastases detection within the liver mask. The liver segmentations have a median Dice similarity coefficient of 0.95 compared with manual annotations. The metastases detection method has a sensitivity of 99.8% with a median of 2 false positives per image. The combination of the two MR sequences in a dual pathway network is proven valuable for the detection of liver metastases. In conclusion, a high quality liver segmentation can be obtained in which we can successfully detect liver metastases. △ Less

Submitted 15 October, 2019; originally announced October 2019.

Journal ref: J. Med. Imag. 6(4), 044003 (2019)

Generalized Richardson-Lucy (GRL) for analyzing multi-shell diffusion MRI data

Authors: Fenghua Guo, Alexander Leemans, Max A. Viergever, Flavio Dell'Acqua, Alberto De Luca

Abstract: Spherical deconvolution is a widely used approach to quantify fiber orientation distribution from diffusion MRI data. The damped Richardson-Lucy (dRL) is developed to perform robust spherical deconvolution on single shell diffusion MRI data. While the dRL algorithm could in theory be directly applied to multi-shell data, it is not optimised to model the signal from multiple tissue types. In this w… ▽ More Spherical deconvolution is a widely used approach to quantify fiber orientation distribution from diffusion MRI data. The damped Richardson-Lucy (dRL) is developed to perform robust spherical deconvolution on single shell diffusion MRI data. While the dRL algorithm could in theory be directly applied to multi-shell data, it is not optimised to model the signal from multiple tissue types. In this work, we introduce a new framework based on dRL - dubbed Generalized Richardson Lucy (GRL) - that uses multi-shell data in combination with user-chosen tissue models to disentangle partial volume effects and increase the accuracy in FOD estimation. The optimal weighting of multi-shell data in the fit and the robustness to noise and partial volume effects of GRL was studied with synthetic data. Subsequently, we investigated the performances of GRL in comparison to dRL on a high-resolution diffusion MRI dataset from the Human Connectome Project and on an MRI dataset acquired at 3T on a clinical scanner. The feasibility of including intra-voxel incoherent motion (IVIM) effects in the modelling was studied on a third dataset. Results of simulations show that GRL can robustly disentangle different tissue types at SNR above 20 and improves the angular accuracy of the FOD estimation. On real data, GRL provides signal fraction maps that are physiologically plausible and consistent between datasets. When considering IVIM effects, high blood pseudo-diffusion fraction is observed in the medial temporal lobe and in the sagittal sinus. In comparison to dRL, GRL provides sharper FODs and less spurious peaks in presence of partial volume effects and results in a better tract termination at the grey/white matter interface or at the outer cortical surface. In conclusion, GRL offers a new modular and flexible framework to perform spherical deconvolution of multi-shell data. △ Less

Submitted 20 April, 2020; v1 submitted 11 October, 2019; originally announced October 2019.

Journal ref: Neuroimage Volume 218, September 2020, 116948

Harmonization of diffusion MRI datasets with adaptive dictionary learning

Authors: Samuel St-Jean, Max A. Viergever, Alexander Leemans

Abstract: Diffusion magnetic resonance imaging is a noninvasive imaging technique that can indirectly infer the microstructure of tissues and provide metrics which are subject to normal variability across subjects. Potentially abnormal values or features may yield essential information to support analysis of controls and patients cohorts, but subtle confounds affecting diffusion MRI, such as those due to di… ▽ More Diffusion magnetic resonance imaging is a noninvasive imaging technique that can indirectly infer the microstructure of tissues and provide metrics which are subject to normal variability across subjects. Potentially abnormal values or features may yield essential information to support analysis of controls and patients cohorts, but subtle confounds affecting diffusion MRI, such as those due to difference in scanning protocols or hardware, can lead to systematic errors which could be mistaken for purely biologically driven variations amongst subjects. In this work, we propose a new harmonization algorithm based on adaptive dictionary learning to mitigate the unwanted variability caused by different scanner hardware while preserving the natural biological variability present in the data. Overcomplete dictionaries, which are learned automatically from the data and do not require paired samples, are then used to reconstruct the data from a different scanner, removing variability present in the source scanner in the process. We use the publicly available database from an international challenge to evaluate the method, which was acquired on three different scanners and with two different protocols, and propose a new map** towards a scanner-agnostic space. Results show that the effect size of the four studied diffusion metrics is preserved while removing variability attributable to the scanner. Experiments with alterations using a free water compartment, which is not simulated in the training data, shows that the effect size induced by the alterations is also preserved after harmonization. The algorithm is freely available and could help multicenter studies in pooling their data, while removing scanner specific confounds, and increase statistical power in the process. △ Less

Submitted 11 June, 2020; v1 submitted 1 October, 2019; originally announced October 2019.

Comments: v5 Peer review for Human Brain Map** v4: Peer review round 2 v3: Peer reviewed version v2: Fix minor text issue + add supp materials v1: To be submitted to Neuroimage

Journal ref: Human Brain Map**, hbm. 25117 (2020)

SMART tracking: Simultaneous anatomical imaging and real-time passive device tracking for MR-guided interventions

Authors: Frank Zijlstra, Max A. Viergever, Peter R. Seevinck

Abstract: Purpose: This study demonstrates a proof of concept of a method for simultaneous anatomical imaging and real-time (SMART) passive device tracking for MR-guided interventions. Methods: Phase Correlation template matching was combined with a fast undersampled radial multi-echo acquisition using the white marker phenomenon after the first echo. In this way, the first echo provides anatomical contra… ▽ More Purpose: This study demonstrates a proof of concept of a method for simultaneous anatomical imaging and real-time (SMART) passive device tracking for MR-guided interventions. Methods: Phase Correlation template matching was combined with a fast undersampled radial multi-echo acquisition using the white marker phenomenon after the first echo. In this way, the first echo provides anatomical contrast, whereas the other echoes provide white marker contrast to allow accurate device localization using fast simulations and template matching. This approach was tested on tracking of five 0.5 mm steel markers in an agarose phantom and on insertion of an MRI-compatible 20 Gauge titanium needle in ex vivo porcine tissue. The locations of the steel markers were quantitatively compared to the marker locations as found on a CT scan of the same phantom. Results: The average pairwise error between the MRI and CT locations was 0.30 mm for tracking of stationary steel spheres and 0.29 mm during motion. Qualitative evaluation of the tracking of needle insertions showed that tracked positions were stable throughout needle insertion and retraction. Conclusions: The proposed SMART tracking method provided accurate passive tracking of devices at high framerates, inclusion of real-time anatomical scanning, and the capability of automatic slice positioning. Furthermore, the method does not require specialized hardware and could therefore be applied to track any rigid metal device that causes appreciable magnetic field distortions. △ Less

Submitted 28 August, 2019; originally announced August 2019.

Journal ref: Physica Medica 64 (2019): 252-260

Automatic brain tissue segmentation in fetal MRI using convolutional neural networks

Authors: N. Khalili, N. Lessmann, E. Turk, N. Claessens, R. de Heus, T. Kolk, M. A. Viergever, M. J. N. L. Benders, I. Isgum

Abstract: MR images of fetuses allow clinicians to detect brain abnormalities in an early stage of development. The cornerstone of volumetric and morphologic analysis in fetal MRI is segmentation of the fetal brain into different tissue classes. Manual segmentation is cumbersome and time consuming, hence automatic segmentation could substantially simplify the procedure. However, automatic brain tissue segme… ▽ More MR images of fetuses allow clinicians to detect brain abnormalities in an early stage of development. The cornerstone of volumetric and morphologic analysis in fetal MRI is segmentation of the fetal brain into different tissue classes. Manual segmentation is cumbersome and time consuming, hence automatic segmentation could substantially simplify the procedure. However, automatic brain tissue segmentation in these scans is challenging owing to artifacts including intensity inhomogeneity, caused in particular by spontaneous fetal movements during the scan. Unlike methods that estimate the bias field to remove intensity inhomogeneity as a preprocessing step to segmentation, we propose to perform segmentation using a convolutional neural network that exploits images with synthetically introduced intensity inhomogeneity as data augmentation. The method first uses a CNN to extract the intracranial volume. Thereafter, another CNN with the same architecture is employed to segment the extracted volume into seven brain tissue classes: cerebellum, basal ganglia and thalami, ventricular cerebrospinal fluid, white matter, brain stem, cortical gray matter and extracerebral cerebrospinal fluid. To make the method applicable to slices showing intensity inhomogeneity artifacts, the training data was augmented by applying a combination of linear gradients with random offsets and orientations to image slices without artifacts. △ Less

Submitted 11 June, 2019; originally announced June 2019.

Comments: Published in Magnetic Resonance Imaging, 2019

Generative adversarial network for segmentation of motion affected neonatal brain MRI

Authors: N. Khalili, E. Turk, M. Zreik, M. A. Viergever, M. J. N. L. Benders, I. Isgum

Abstract: Automatic neonatal brain tissue segmentation in preterm born infants is a prerequisite for evaluation of brain development. However, automatic segmentation is often hampered by motion artifacts caused by infant head movements during image acquisition. Methods have been developed to remove or minimize these artifacts during image reconstruction using frequency domain data. However, frequency domain… ▽ More Automatic neonatal brain tissue segmentation in preterm born infants is a prerequisite for evaluation of brain development. However, automatic segmentation is often hampered by motion artifacts caused by infant head movements during image acquisition. Methods have been developed to remove or minimize these artifacts during image reconstruction using frequency domain data. However, frequency domain data might not always be available. Hence, in this study we propose a method for removing motion artifacts from the already reconstructed MR scans. The method employs a generative adversarial network trained with a cycle consistency loss to transform slices affected by motion into slices without motion artifacts, and vice versa. In the experiments 40 T2-weighted coronal MR scans of preterm born infants imaged at 30 weeks postmenstrual age were used. All images contained slices affected by motion artifacts hampering automatic tissue segmentation. To evaluate whether correction allows more accurate image segmentation, the images were segmented into 8 tissue classes: cerebellum, myelinated white matter, basal ganglia and thalami, ventricular cerebrospinal fluid, white matter, brain stem, cortical gray matter, and extracerebral cerebrospinal fluid. Images corrected for motion and corresponding segmentations were qualitatively evaluated using 5-point Likert scale. Before the correction of motion artifacts, median image quality and quality of corresponding automatic segmentations were assigned grade 2 (poor) and 3 (moderate), respectively. After correction of motion artifacts, both improved to grades 3 and 4, respectively. The results indicate that correction of motion artifacts in the image space using the proposed approach allows accurate segmentation of brain tissue classes in slices affected by motion artifacts. △ Less

Submitted 11 June, 2019; originally announced June 2019.

Comments: Accepted in Medical Image Computing and Computer Assisted Intervention 2019

Deep learning analysis of coronary arteries in cardiac CT angiography for detection of patients requiring invasive coronary angiography

Authors: Majd Zreik, Robbert W. van Hamersvelt, Nadieh Khalili, Jelmer M. Wolterink, Michiel Voskuil, Max A. Viergever, Tim Leiner, Ivana Išgum

Abstract: In patients with obstructive coronary artery disease, the functional significance of a coronary artery stenosis needs to be determined to guide treatment. This is typically established through fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA). We present a method for automatic and non-invasive detection of patients requiring ICA, employing deep unsuper… ▽ More In patients with obstructive coronary artery disease, the functional significance of a coronary artery stenosis needs to be determined to guide treatment. This is typically established through fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA). We present a method for automatic and non-invasive detection of patients requiring ICA, employing deep unsupervised analysis of complete coronary arteries in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA scans of 187 patients, 137 of them underwent invasive FFR measurement in 192 different coronary arteries. These FFR measurements served as a reference standard for the functional significance of the coronary stenosis. The centerlines of the coronary arteries were extracted and used to reconstruct straightened multi-planar reformatted (MPR) volumes. To automatically identify arteries with functionally significant stenosis that require ICA, each MPR volume was encoded into a fixed number of encodings using two disjoint 3D and 1D convolutional autoencoders performing spatial and sequential encodings, respectively. Thereafter, these encodings were employed to classify arteries using a support vector machine classifier. The detection of coronary arteries requiring invasive evaluation, evaluated using repeated cross-validation experiments, resulted in an area under the receiver operating characteristic curve of $0.81 \pm 0.02$ on the artery-level, and $0.87 \pm 0.02$ on the patient-level. The results demonstrate the feasibility of automatic non-invasive detection of patients that require ICA and possibly subsequent coronary artery intervention. This could potentially reduce the number of patients that unnecessarily undergo ICA. △ Less

Submitted 10 November, 2019; v1 submitted 11 June, 2019; originally announced June 2019.

Comments: This work has been accepted to IEEE TMI for publication

Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge

Authors: Hugo J. Kuijf, J. Matthijs Biesbroek, Jeroen de Bresser, Rutger Heinen, Simon Andermatt, Mariana Bento, Matt Berseth, Mikhail Belyaev, M. Jorge Cardoso, Adrià Casamitjana, D. Louis Collins, Mahsa Dadar, Achilleas Georgiou, Mohsen Ghafoorian, Dakai **, April Khademi, Jesse Knight, Hongwei Li, Xavier Lladó, Miguel Luna, Qaiser Mahmood, Richard McKinley, Alireza Mehrtash, Sébastien Ourselin, Bo-yong Park , et al. (19 additional authors not shown)

Abstract: Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. Automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We… ▽ More Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. Automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their method on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge (https://wmh.isi.uu.nl/). Sixty T1+FLAIR images from three MR scanners were released with manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. Segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: (1) Dice similarity coefficient, (2) modified Hausdorff distance (95th percentile), (3) absolute log-transformed volume difference, (4) sensitivity for detecting individual lesions, and (5) F1-score for individual lesions. Additionally, methods were ranked on their inter-scanner robustness. Twenty participants submitted their method for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all methods generalize to unseen scanners. The challenge remains open for future submissions and provides a public platform for method evaluation. △ Less

Submitted 1 April, 2019; originally announced April 2019.

Comments: Accepted for publication in IEEE Transactions on Medical Imaging

Coronary Artery Centerline Extraction in Cardiac CT Angiography Using a CNN-Based Orientation Classifier

Authors: Jelmer M. Wolterink, Robbert W. van Hamersvelt, Max A. Viergever, Tim Leiner, Ivana Išgum

Abstract: Coronary artery centerline extraction in cardiac CT angiography (CCTA) images is a prerequisite for evaluation of stenoses and atherosclerotic plaque. We propose an algorithm that extracts coronary artery centerlines in CCTA using a convolutional neural network (CNN). A 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based… ▽ More Coronary artery centerline extraction in cardiac CT angiography (CCTA) images is a prerequisite for evaluation of stenoses and atherosclerotic plaque. We propose an algorithm that extracts coronary artery centerlines in CCTA using a convolutional neural network (CNN). A 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based on a local image patch. Starting from a single seed point placed manually or automatically anywhere in a coronary artery, a tracker follows the vessel centerline in two directions using the predictions of the CNN. Tracking is terminated when no direction can be identified with high certainty. The CNN was trained using 32 manually annotated centerlines in a training set consisting of 8 CCTA images provided in the MICCAI 2008 Coronary Artery Tracking Challenge (CAT08). Evaluation using 24 test images of the CAT08 challenge showed that extracted centerlines had an average overlap of 93.7% with 96 manually annotated reference centerlines. Extracted centerline points were highly accurate, with an average distance of 0.21 mm to reference centerline points. In a second test set consisting of 50 CCTA scans, 5,448 markers in the coronary arteries were used as seed points to extract single centerlines. This showed strong correspondence between extracted centerlines and manually placed markers. In a third test set containing 36 CCTA scans, fully automatic seeding and centerline extraction led to extraction of on average 92% of clinically relevant coronary artery segments. The proposed method is able to accurately and efficiently determine the direction and radius of coronary arteries. The method can be trained with limited training data, and once trained allows fast automatic or interactive extraction of coronary artery trees from CCTA images. △ Less

Submitted 24 October, 2018; v1 submitted 7 October, 2018; originally announced October 2018.

Comments: Accepted in Medical Image Analysis

Direct Prediction of Cardiovascular Mortality from Low-dose Chest CT using Deep Learning

Authors: Sanne G. M. van Velzen, Majd Zreik, Nikolas Lessmann, Max A. Viergever, Pim A. de Jong, Helena M. Verkooijen, Ivana Išgum

Abstract: Cardiovascular disease (CVD) is a leading cause of death in the lung cancer screening population. Chest CT scans made in lung cancer screening are suitable for identification of participants at risk of CVD. Existing methods analyzing CT images from lung cancer screening for prediction of CVD events or mortality use engineered features extracted from the images combined with patient information. In… ▽ More Cardiovascular disease (CVD) is a leading cause of death in the lung cancer screening population. Chest CT scans made in lung cancer screening are suitable for identification of participants at risk of CVD. Existing methods analyzing CT images from lung cancer screening for prediction of CVD events or mortality use engineered features extracted from the images combined with patient information. In this work we propose a method that automatically predicts 5-year cardiovascular mortality directly from chest CT scans without the need for hand-crafting image features. A set of 1,583 participants of the National Lung Screening Trial was included (1,188 survivors, 395 non-survivors). Low-dose chest CT images acquired at baseline were analyzed and the follow-up time was 5 years. To limit the analysis to the heart region, the heart was first localized by our previously developed algorithm for organ localization exploiting convolutional neural networks. Thereafter, a convolutional autoencoder was used to encode the identified heart region. Finally, based on the extracted encodings subjects were classified into survivors or non-survivors using a support vector machine classifier. The performance of the method was assessed in eight cross-validation experiments with 1,433 images used for training, 50 for validation and 100 for testing. The method achieved a performance with an area under the ROC curve of 0.72. The results demonstrate that prediction of cardiovascular mortality directly from low-dose screening chest CT scans, without hand-crafted features, is feasible, allowing identification of subjects at risk of fatal CVD events. △ Less

Submitted 4 October, 2018; originally announced October 2018.

Comments: This work has been submitted to SPIE 2019 conference

A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration

Authors: Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Hessam Sokooti, Marius Staring, Ivana Isgum

Abstract: Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtainin… ▽ More Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for \textit{unsupervised} affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster. △ Less

Submitted 5 December, 2018; v1 submitted 17 September, 2018; originally announced September 2018.

Comments: Accepted: Medical Image Analysis - Elsevier

Journal ref: Medical Image Analysis, Volume 52, February 2019, Pages 128-143

Automatic, fast and robust characterization of noise distributions for diffusion MRI

Authors: Samuel St-Jean, Alberto De Luca, Max A. Viergever, Alexander Leemans

Abstract: Knowledge of the noise distribution in magnitude diffusion MRI images is the centerpiece to quantify uncertainties arising from the acquisition process. The use of parallel imaging methods, the number of receiver coils and imaging filters applied by the scanner, amongst other factors, dictate the resulting signal distribution. Accurate estimation beyond textbook Rician or noncentral chi distributi… ▽ More Knowledge of the noise distribution in magnitude diffusion MRI images is the centerpiece to quantify uncertainties arising from the acquisition process. The use of parallel imaging methods, the number of receiver coils and imaging filters applied by the scanner, amongst other factors, dictate the resulting signal distribution. Accurate estimation beyond textbook Rician or noncentral chi distributions often requires information about the acquisition process (e.g. coils sensitivity maps or reconstruction coefficients), which is not usually available. We introduce a new method where a change of variable naturally gives rise to a particular form of the gamma distribution for background signals. The first moments and maximum likelihood estimators of this gamma distribution explicitly depend on the number of coils, making it possible to estimate all unknown parameters using only the magnitude data. A rejection step is used to make the method automatic and robust to artifacts. Experiments on synthetic datasets show that the proposed method can reliably estimate both the degrees of freedom and the standard deviation. The worst case errors range from below 2% (spatially uniform noise) to approximately 10% (spatially variable noise). Repeated acquisitions of in vivo datasets show that the estimated parameters are stable and have lower variances than compared methods. △ Less

Submitted 2 October, 2018; v1 submitted 30 May, 2018; originally announced May 2018.

Comments: v2: added publisher DOI statement, fixed text typo in appendix A2

Journal ref: St-Jean S. et al. (2018) Automatic, Fast and Robust Characterization of Noise Distributions for Diffusion MRI. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2018. LNCS, vol 11070. Springer, Cham

A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography

Authors: Majd Zreik, Robbert W. van Hamersvelt, Jelmer M. Wolterink, Tim Leiner, Max A. Viergever, Ivana Isgum

Abstract: Various types of atherosclerotic plaque and varying grades of stenosis could lead to different management of patients with coronary artery disease. Therefore, it is crucial to detect and classify the type of coronary artery plaque, as well as to detect and determine the degree of coronary artery stenosis. This study includes retrospectively collected clinically obtained coronary CT angiography (CC… ▽ More Various types of atherosclerotic plaque and varying grades of stenosis could lead to different management of patients with coronary artery disease. Therefore, it is crucial to detect and classify the type of coronary artery plaque, as well as to detect and determine the degree of coronary artery stenosis. This study includes retrospectively collected clinically obtained coronary CT angiography (CCTA) scans of 163 patients. To perform automatic analysis for coronary artery plaque and stenosis classification, a multi-task recurrent convolutional neural network is applied on multi-planar reformatted (MPR) images of the coronary arteries. First, a 3D convolutional neural network is utilized to extract features along the coronary artery. Subsequently, the extracted features are aggregated by a recurrent neural network that performs two simultaneous multi-class classification tasks. In the first task, the network detects and characterizes the type of the coronary artery plaque (no plaque, non-calcified, mixed, calcified). In the second task, the network detects and determines the anatomical significance of the coronary artery stenosis (no stenosis, non-significant i.e. <50% luminal narrowing, significant i.e. >50% luminal narrowing). For detection and classification of coronary plaque, the method achieved an accuracy of 0.77. For detection and classification of stenosis, the method achieved an accuracy of 0.80. The results demonstrate that automatic detection and classification of coronary artery plaque and stenosis are feasible. This may enable automated triage of patients to those without coronary plaque and those with coronary plaque and stenosis in need for further cardiovascular workup. △ Less

Submitted 10 December, 2018; v1 submitted 12 April, 2018; originally announced April 2018.

Comments: Published in IEEE Transactions on Medical Imaging, 2019

Direct and Real-Time Cardiovascular Risk Prediction

Authors: Bob D. de Vos, Nikolas Lessmann, Pim A. de Jong, Max A. Viergever, Ivana Isgum

Abstract: Coronary artery calcium (CAC) burden quantified in low-dose chest CT is a predictor of cardiovascular events. We propose an automatic method for CAC quantification, circumventing intermediate segmentation of CAC. The method determines a bounding box around the heart using a ConvNet for localization. Subsequently, a dedicated ConvNet analyzes axial slices within the bounding boxes to determine CAC… ▽ More Coronary artery calcium (CAC) burden quantified in low-dose chest CT is a predictor of cardiovascular events. We propose an automatic method for CAC quantification, circumventing intermediate segmentation of CAC. The method determines a bounding box around the heart using a ConvNet for localization. Subsequently, a dedicated ConvNet analyzes axial slices within the bounding boxes to determine CAC quantity by regression. A dataset of 1,546 baseline CT scans was used from the National Lung Screening Trial with manually identified CAC. The method achieved an ICC of 0.98 between manual reference and automatically obtained Agatston scores. Stratification of subjects into five cardiovascular risk categories resulted in an accuracy of 85\% and Cohen's linearly weighted $κ$ of 0.90. The results demonstrate that real-time quantification of CAC burden in chest CT without the need for segmentation of CAC is possible. △ Less

Submitted 8 December, 2017; originally announced December 2017.

Comments: Scientific paper at RSNA 2017

Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis

Authors: Majd Zreik, Nikolas Lessmann, Robbert W. van Hamersvelt, Jelmer M. Wolterink, Michiel Voskuil, Max A. Viergever, Tim Leiner, Ivana Išgum

Abstract: In patients with coronary artery stenoses of intermediate severity, the functional significance needs to be determined. Fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA), is most often used in clinical practice. To reduce the number of ICA procedures, we present a method for automatic identification of patients with functionally significant coronary ar… ▽ More In patients with coronary artery stenoses of intermediate severity, the functional significance needs to be determined. Fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA), is most often used in clinical practice. To reduce the number of ICA procedures, we present a method for automatic identification of patients with functionally significant coronary artery stenoses, employing deep learning analysis of the left ventricle (LV) myocardium in rest coronary CT angiography (CCTA). The study includes consecutively acquired CCTA scans of 166 patients with FFR measurements. To identify patients with a functionally significant coronary artery stenosis, analysis is performed in several stages. First, the LV myocardium is segmented using a multiscale convolutional neural network (CNN). To characterize the segmented LV myocardium, it is subsequently encoded using unsupervised convolutional autoencoder (CAE). Thereafter, patients are classified according to the presence of functionally significant stenosis using an SVM classifier based on the extracted and clustered encodings. Quantitative evaluation of LV myocardium segmentation in 20 images resulted in an average Dice coefficient of 0.91 and an average mean absolute distance between the segmented and reference LV boundaries of 0.7 mm. Classification of patients was evaluated in the remaining 126 CCTA scans in 50 10-fold cross-validation experiments and resulted in an area under the receiver operating characteristic curve of 0.74 +- 0.02. At sensitivity levels 0.60, 0.70 and 0.80, the corresponding specificity was 0.77, 0.71 and 0.59, respectively. The results demonstrate that automatic analysis of the LV myocardium in a single CCTA scan acquired at rest, without assessment of the anatomy of the coronary arteries, can be used to identify patients with functionally significant coronary artery stenosis. △ Less

Submitted 6 December, 2017; v1 submitted 24 November, 2017; originally announced November 2017.

Comments: This paper was submitted in April 2017 and accepted in November 2017 for publication in Medical Image Analysis. Please cite as: Zreik et al., Medical Image Analysis, 2018, vol. 44, pp. 72-85

Automatic calcium scoring in low-dose chest CT using deep neural networks with dilated convolutions

Authors: Nikolas Lessmann, Bram van Ginneken, Majd Zreik, Pim A. de Jong, Bob D. de Vos, Max A. Viergever, Ivana Išgum

Abstract: Heavy smokers undergoing screening with low-dose chest CT are affected by cardiovascular disease as much as by lung cancer. Low-dose chest CT scans acquired in screening enable quantification of atherosclerotic calcifications and thus enable identification of subjects at increased cardiovascular risk. This paper presents a method for automatic detection of coronary artery, thoracic aorta and cardi… ▽ More Heavy smokers undergoing screening with low-dose chest CT are affected by cardiovascular disease as much as by lung cancer. Low-dose chest CT scans acquired in screening enable quantification of atherosclerotic calcifications and thus enable identification of subjects at increased cardiovascular risk. This paper presents a method for automatic detection of coronary artery, thoracic aorta and cardiac valve calcifications in low-dose chest CT using two consecutive convolutional neural networks. The first network identifies and labels potential calcifications according to their anatomical location and the second network identifies true calcifications among the detected candidates. This method was trained and evaluated on a set of 1744 CT scans from the National Lung Screening Trial. To determine whether any reconstruction or only images reconstructed with soft tissue filters can be used for calcification detection, we evaluated the method on soft and medium/sharp filter reconstructions separately. On soft filter reconstructions, the method achieved F1 scores of 0.89, 0.89, 0.67, and 0.55 for coronary artery, thoracic aorta, aortic valve and mitral valve calcifications, respectively. On sharp filter reconstructions, the F1 scores were 0.84, 0.81, 0.64, and 0.66, respectively. Linearly weighted kappa coefficients for risk category assignment based on per subject coronary artery calcium were 0.91 and 0.90 for soft and sharp filter reconstructions, respectively. These results demonstrate that the presented method enables reliable automatic cardiovascular risk assessment in all low-dose chest CT scans acquired for lung cancer screening. △ Less

Submitted 1 February, 2018; v1 submitted 1 November, 2017; originally announced November 2017.

Journal ref: IEEE Transactions on Medical Imaging 37(2), pp 615-625, 2018

Automatic segmentation of the intracranialvolume in fetal MR images

Authors: N. Khalili, P. Moeskops, N. H. P. Claessens, S. Scherpenzeel, E. Turk, R. de Heus, M. J. N. L. Benders, M. A. Viergever, J. P. W. Pluim, I. Išgum

Abstract: MR images of the fetus allow non-invasive analysis of the fetal brain. Quantitative analysis of fetal brain development requires automatic brain tissue segmentation that is typically preceded by segmentation of the intracranial volume (ICV). This is challenging because fetal MR images visualize the whole moving fetus and in addition partially visualize the maternal body. This paper presents an aut… ▽ More MR images of the fetus allow non-invasive analysis of the fetal brain. Quantitative analysis of fetal brain development requires automatic brain tissue segmentation that is typically preceded by segmentation of the intracranial volume (ICV). This is challenging because fetal MR images visualize the whole moving fetus and in addition partially visualize the maternal body. This paper presents an automatic method for segmentation of the ICV in fetal MR images. The method employs a multi-scale convolutional neural network in 2D slices to enable learning spatial information from larger context as well as detailed local information. The method is developed and evaluated with 30 fetal T2-weighted MRI scans (average age $33.2\pm1.2$ weeks postmenstrual age). The set contains $10$ scans acquired in axial, $10$ in coronal and $10$ in sagittal imaging planes. A reference standard was defined in all images by manual annotation of the intracranial volume in $10$ equidistantly distributed slices. The automatic analysis was performed by training and testing the network using scans acquired in the representative imaging plane as well as combining the training data from all imaging planes. On average, the automatic method achieved Dice coefficients of 0.90 for the axial images, 0.90 for the coronal images and 0.92 for the sagittal images. Combining the training sets resulted in average Dice coefficients of 0.91 for the axial images, 0.95 for the coronal images, and 0.92 for the sagittal images. The results demonstrate that the evaluated method achieved good performance in extracting ICV in fetal MR scans regardless of the imaging plane. △ Less

Submitted 31 July, 2017; originally announced August 2017.

Automatic Segmentation and Disease Classification Using Cardiac Cine MR Images

Authors: Jelmer M. Wolterink, Tim Leiner, Max A. Viergever, Ivana Isgum

Abstract: Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features… ▽ More Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features derived from the obtained segmentations were used in a Random Forest classifier to label patients as suffering from dilated cardiomyopathy, hypertrophic cardiomyopathy, heart failure following myocardial infarction, right ventricular abnormality, or no cardiac disease. The method was developed and evaluated using a balanced dataset containing images of 100 patients, which was provided in the MICCAI 2017 automated cardiac diagnosis challenge (ACDC). The segmentation and classification pipeline were evaluated in a four-fold stratified cross-validation. Average Dice scores between reference and automatically obtained segmentations were 0.94, 0.88 and 0.87 for the LV, RV and myocardium. The classifier assigned 91% of patients to the correct disease category. Segmentation and disease classification took 5 s per patient. The results of our study suggest that image-based diagnosis using cine MR cardiac scans can be performed automatically with high accuracy. △ Less

Submitted 3 August, 2017; originally announced August 2017.

Comments: Accepted in STACOM Automated Cardiac Diagnosis Challenge 2017

End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network

Authors: Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Marius Staring, Ivana Išgum

Abstract: In this work we propose a deep learning network for deformable image registration (DIRNet). The DIRNet consists of a convolutional neural network (ConvNet) regressor, a spatial transformer, and a resampler. The ConvNet analyzes a pair of fixed and moving images and outputs parameters for the spatial transformer, which generates the displacement vector field that enables the resampler to warp the m… ▽ More In this work we propose a deep learning network for deformable image registration (DIRNet). The DIRNet consists of a convolutional neural network (ConvNet) regressor, a spatial transformer, and a resampler. The ConvNet analyzes a pair of fixed and moving images and outputs parameters for the spatial transformer, which generates the displacement vector field that enables the resampler to warp the moving image to the fixed image. The DIRNet is trained end-to-end by unsupervised optimization of a similarity metric between input image pairs. A trained DIRNet can be applied to perform registration on unseen image pairs in one pass, thus non-iteratively. Evaluation was performed with registration of images of handwritten digits (MNIST) and cardiac cine MR scans (Sunnybrook Cardiac Data). The results demonstrate that registration with DIRNet is as accurate as a conventional deformable image registration method with substantially shorter execution times. △ Less

Submitted 20 April, 2017; originally announced April 2017.

Journal ref: DLMIA/ML-CDS@MICCAI 2017

Automatic Segmentation of the Left Ventricle in Cardiac CT Angiography Using Convolutional Neural Network

Authors: Majd Zreik, Tim Leiner, Bob D. de Vos, Robbert W. van Hamersvelt, Max A. Viergever, Ivana Isgum

Abstract: Accurate delineation of the left ventricle (LV) is an important step in evaluation of cardiac function. In this paper, we present an automatic method for segmentation of the LV in cardiac CT angiography (CCTA) scans. Segmentation is performed in two stages. First, a bounding box around the LV is detected using a combination of three convolutional neural networks (CNNs). Subsequently, to obtain the… ▽ More Accurate delineation of the left ventricle (LV) is an important step in evaluation of cardiac function. In this paper, we present an automatic method for segmentation of the LV in cardiac CT angiography (CCTA) scans. Segmentation is performed in two stages. First, a bounding box around the LV is detected using a combination of three convolutional neural networks (CNNs). Subsequently, to obtain the segmentation of the LV, voxel classification is performed within the defined bounding box using a CNN. The study included CCTA scans of sixty patients, fifty scans were used to train the CNNs for the LV localization, five scans were used to train LV segmentation and the remaining five scans were used for testing the method. Automatic segmentation resulted in the average Dice coefficient of 0.85 and mean absolute surface distance of 1.1 mm. The results demonstrate that automatic segmentation of the LV in CCTA scans using voxel classification with convolutional neural networks is feasible. △ Less

Submitted 19 April, 2017; originally announced April 2017.

Comments: This work has been published as: Zreik, M., Leiner, T., de Vos, B. D., van Hamersvelt, R. W., Viergever, M. A., Išgum, I. (2016, April). Automatic segmentation of the left ventricle in cardiac CT angiography using convolutional neural networks. In Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on (pp. 40-43). IEEE

ConvNet-Based Localization of Anatomical Structures in 3D Medical Images

Authors: Bob D. de Vos, Jelmer M. Wolterink, Pim A. de Jong, Tim Leiner, Max A. Viergever, Ivana Išgum

Abstract: Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet). A single ConvNet is trained to detect presence of the anatomical structure of interest in ax… ▽ More Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet). A single ConvNet is trained to detect presence of the anatomical structure of interest in axial, coronal, and sagittal slices extracted from a 3D image. To allow the ConvNet to analyze slices of different sizes, spatial pyramid pooling is applied. After detection, 3D bounding boxes are created by combining the output of the ConvNet in all slices. In the experiments 200 chest CT, 100 cardiac CT angiography (CTA), and 100 abdomen CT scans were used. The heart, ascending aorta, aortic arch, and descending aorta were localized in chest CT scans, the left cardiac ventricle in cardiac CTA scans, and the liver in abdomen CT scans. Localization was evaluated using the distances between automatically and manually defined reference bounding box centroids and walls. The best results were achieved in localization of structures with clearly defined boundaries (e.g. aortic arch) and the worst when the structure boundary was not clearly visible (e.g. liver). The method was more robust and accurate in localization multiple structures. △ Less

Submitted 19 April, 2017; originally announced April 2017.

Journal ref: IEEE Transactions on Medical Imaging , vol.PP, no.99, pp.1-1 (2017)

Dilated Convolutional Neural Networks for Cardiovascular MR Segmentation in Congenital Heart Disease

Authors: Jelmer M. Wolterink, Tim Leiner, Max A. Viergever, Ivana Išgum

Abstract: We propose an automatic method using dilated convolutional neural networks (CNNs) for segmentation of the myocardium and blood pool in cardiovascular MR (CMR) of patients with congenital heart disease (CHD). Ten training and ten test CMR scans cropped to an ROI around the heart were provided in the MICCAI 2016 HVSMR challenge. A dilated CNN with a receptive field of 131x131 voxels was trained fo… ▽ More We propose an automatic method using dilated convolutional neural networks (CNNs) for segmentation of the myocardium and blood pool in cardiovascular MR (CMR) of patients with congenital heart disease (CHD). Ten training and ten test CMR scans cropped to an ROI around the heart were provided in the MICCAI 2016 HVSMR challenge. A dilated CNN with a receptive field of 131x131 voxels was trained for myocardium and blood pool segmentation in axial, sagittal and coronal image slices. Performance was evaluated within the HVSMR challenge. Automatic segmentation of the test scans resulted in Dice indices of 0.80$\pm$0.06 and 0.93$\pm$0.02, average distances to boundaries of 0.96$\pm$0.31 and 0.89$\pm$0.24 mm, and Hausdorff distances of 6.13$\pm$3.76 and 7.07$\pm$3.01 mm for the myocardium and blood pool, respectively. Segmentation took 41.5$\pm$14.7 s per scan. In conclusion, dilated CNNs trained on a small set of CMR images of CHD patients showing large anatomical variability provide accurate myocardium and blood pool segmentations. △ Less

Submitted 12 April, 2017; originally announced April 2017.

Journal ref: RAMBO 2016, HVSMR 2016. LNCS 10129. pp. 95-102

Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities

Authors: Pim Moeskops, Jelmer M. Wolterink, Bas H. M. van der Velden, Kenneth G. A. Gilhuijs, Tim Leiner, Max A. Viergever, Ivana Išgum

Abstract: Automatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single convolutional neural network (CNN) can be trained to perform different segmentation tasks. A single CNN is trained to segment six tissues in MR brain image… ▽ More Automatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single convolutional neural network (CNN) can be trained to perform different segmentation tasks. A single CNN is trained to segment six tissues in MR brain images, the pectoral muscle in MR breast images, and the coronary arteries in cardiac CTA. The CNN therefore learns to identify the imaging modality, the visualised anatomical structures, and the tissue classes. For each of the three tasks (brain MRI, breast MRI and cardiac CTA), this combined training procedure resulted in a segmentation performance equivalent to that of a CNN trained specifically for that task, demonstrating the high capacity of CNN architectures. Hence, a single system could be used in clinical practice to automatically perform diverse segmentation tasks without task-specific training. △ Less

Submitted 11 April, 2017; originally announced April 2017.

Journal ref: Moeskops, P., Wolterink, J.M., van der Velden, B.H.M., Gilhuijs, K.G.A., Leiner, T., Viergever, M.A., Išgum, I. Deep learning for multi-task medical image segmentation in multiple modalities. In: MICCAI 2016, pp. 478-486

Automatic segmentation of MR brain images with a convolutional neural network

Authors: Pim Moeskops, Max A. Viergever, Adriënne M. Mendrik, Linda S. de Vries, Manon J. N. L. Benders, Ivana Išgum

Abstract: Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network use… ▽ More Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the information that is important for the classification based on training data. The method requires a single anatomical MR image only. The segmentation method is applied to five different data sets: coronal T2-weighted images of preterm infants acquired at 30 weeks postmenstrual age (PMA) and 40 weeks PMA, axial T2- weighted images of preterm infants acquired at 40 weeks PMA, axial T1-weighted images of ageing adults acquired at an average age of 70 years, and T1-weighted images of young adults acquired at an average age of 23 years. The method obtained the following average Dice coefficients over all segmented tissue classes for each data set, respectively: 0.87, 0.82, 0.84, 0.86 and 0.91. The results demonstrate that the method obtains accurate segmentations in all five sets, and hence demonstrates its robustness to differences in age and acquisition protocol. △ Less

Submitted 11 April, 2017; originally announced April 2017.

Journal ref: IEEE Transactions on Medical Imaging, 35(5), 1252-1261 (2016)

Assessment of algorithms for mitosis detection in breast cancer histopathology images

Authors: Mitko Veta, Paul J. van Diest, Stefan M. Willems, Haibo Wang, Anant Madabhushi, Angel Cruz-Roa, Fabio Gonzalez, Anders B. L. Larsen, Jacob S. Vestergaard, Anders B. Dahl, Dan C. Cireşan, Jürgen Schmidhuber, Alessandro Giusti, Luca M. Gambardella, F. Boray Tek, Thomas Walter, Ching-Wei Wang, Satoshi Kondo, Bogdan J. Matuszewski, Frederic Precioso, Violet Snell, Josef Kittler, Teofilo E. de Campos, Adnan M. Khan, Nasir M. Rajpoot , et al. (4 additional authors not shown)

Abstract: The proliferative activity of breast tumors, which is routinely estimated by counting of mitotic figures in hematoxylin and eosin stained histology sections, is considered to be one of the most important prognostic markers. However, mitosis counting is laborious, subjective and may suffer from low inter-observer agreement. With the wider acceptance of whole slide images in pathology labs, automati… ▽ More The proliferative activity of breast tumors, which is routinely estimated by counting of mitotic figures in hematoxylin and eosin stained histology sections, is considered to be one of the most important prognostic markers. However, mitosis counting is laborious, subjective and may suffer from low inter-observer agreement. With the wider acceptance of whole slide images in pathology labs, automatic image analysis has been proposed as a potential solution for these issues. In this paper, the results from the Assessment of Mitosis Detection Algorithms 2013 (AMIDA13) challenge are described. The challenge was based on a data set consisting of 12 training and 11 testing subjects, with more than one thousand annotated mitotic figures by multiple observers. Short descriptions and results from the evaluation of eleven methods are presented. The top performing method has an error rate that is comparable to the inter-observer agreement among pathologists. △ Less

Submitted 21 November, 2014; originally announced November 2014.

Comments: 23 pages, 5 figures, accepted for publication in the journal Medical Image Analysis