-
From algorithms to action: improving patient care requires causality
Authors:
Wouter A. C. van Amsterdam,
Pim A. de Jong,
Joost J. C. Verhoeff,
Tim Leiner,
Rajesh Ranganath
Abstract:
In cancer research there is much interest in building and validating outcome predicting outcomes to support treatment decisions. However, because most outcome prediction models are developed and validated without regard to the causal aspects of treatment decision making, many published outcome prediction models may cause harm when used for decision making, despite being found accurate in validatio…
▽ More
In cancer research there is much interest in building and validating outcome predicting outcomes to support treatment decisions. However, because most outcome prediction models are developed and validated without regard to the causal aspects of treatment decision making, many published outcome prediction models may cause harm when used for decision making, despite being found accurate in validation studies. Guidelines on prediction model validation and the checklist for risk model endorsement by the American Joint Committee on Cancer do not protect against prediction models that are accurate during development and validation but harmful when used for decision making. We explain why this is the case and how to build and validate models that are useful for decision making.
△ Less
Submitted 1 April, 2024; v1 submitted 15 September, 2022;
originally announced September 2022.
-
AI-based Reconstruction for Fast MRI -- A Systematic Review and Meta-analysis
Authors:
Yutong Chen,
Carola-Bibiane Schönlieb,
Pietro Liò,
Tim Leiner,
Pier Luigi Dragotti,
Ge Wang,
Daniel Rueckert,
David Firmin,
Guang Yang
Abstract:
Compressed sensing (CS) has been playing a key role in accelerating the magnetic resonance imaging (MRI) acquisition process. With the resurgence of artificial intelligence, deep neural networks and CS algorithms are being integrated to redefine the state of the art of fast MRI. The past several years have witnessed substantial growth in the complexity, diversity, and performance of deep learning-…
▽ More
Compressed sensing (CS) has been playing a key role in accelerating the magnetic resonance imaging (MRI) acquisition process. With the resurgence of artificial intelligence, deep neural networks and CS algorithms are being integrated to redefine the state of the art of fast MRI. The past several years have witnessed substantial growth in the complexity, diversity, and performance of deep learning-based CS techniques that are dedicated to fast MRI. In this meta-analysis, we systematically review the deep learning-based CS techniques for fast MRI, describe key model designs, highlight breakthroughs, and discuss promising directions. We have also introduced a comprehensive analysis framework and a classification system to assess the pivotal role of deep learning in CS-based acceleration for MRI.
△ Less
Submitted 23 December, 2021;
originally announced December 2021.
-
Deep Learning from Dual-Energy Information for Whole-Heart Segmentation in Dual-Energy and Single-Energy Non-Contrast-Enhanced Cardiac CT
Authors:
Steffen Bruns,
Jelmer M. Wolterink,
Richard A. P. Takx,
Robbert W. van Hamersvelt,
Dominika Suchá,
Max A. Viergever,
Tim Leiner,
Ivana Išgum
Abstract:
Deep learning-based whole-heart segmentation in coronary CT angiography (CCTA) allows the extraction of quantitative imaging measures for cardiovascular risk prediction. Automatic extraction of these measures in patients undergoing only non-contrast-enhanced CT (NCCT) scanning would be valuable. In this work, we leverage information provided by a dual-layer detector CT scanner to obtain a referenc…
▽ More
Deep learning-based whole-heart segmentation in coronary CT angiography (CCTA) allows the extraction of quantitative imaging measures for cardiovascular risk prediction. Automatic extraction of these measures in patients undergoing only non-contrast-enhanced CT (NCCT) scanning would be valuable. In this work, we leverage information provided by a dual-layer detector CT scanner to obtain a reference standard in virtual non-contrast (VNC) CT images mimicking NCCT images, and train a 3D convolutional neural network (CNN) for the segmentation of VNC as well as NCCT images. Contrast-enhanced acquisitions on a dual-layer detector CT scanner were reconstructed into a CCTA and a perfectly aligned VNC image. In each CCTA image, manual reference segmentations of the left ventricular (LV) myocardium, LV cavity, right ventricle, left atrium, right atrium, ascending aorta, and pulmonary artery trunk were obtained and propagated to the corresponding VNC image. These VNC images and reference segmentations were used to train 3D CNNs for automatic segmentation in either VNC images or NCCT images. Automatic segmentations in VNC images showed good agreement with reference segmentations, with an average Dice similarity coefficient of 0.897 \pm 0.034 and an average symmetric surface distance of 1.42 \pm 0.45 mm. Volume differences [95% confidence interval] between automatic NCCT and reference CCTA segmentations were -19 [-67; 30] mL for LV myocardium, -25 [-78; 29] mL for LV cavity, -29 [-73; 14] mL for right ventricle, -20 [-62; 21] mL for left atrium, and -19 [-73; 34] mL for right atrium, respectively. In 214 (74%) NCCT images from an independent multi-vendor multi-center set, two observers agreed that the automatic segmentation was mostly accurate or better. This method might enable quantification of additional cardiac measures from NCCT images for improved cardiovascular risk prediction.
△ Less
Submitted 10 August, 2020;
originally announced August 2020.
-
Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images
Authors:
Julia M. H. Noothout,
Bob D. de Vos,
Jelmer M. Wolterink,
Elbrich M. Postma,
Paul A. M. Smeets,
Richard A. P. Takx,
Tim Leiner,
Max A. Viergever,
Ivana Išgum
Abstract:
In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vector…
▽ More
In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage.
△ Less
Submitted 10 July, 2020;
originally announced July 2020.
-
Combined analysis of coronary arteries and the left ventricular myocardium in cardiac CT angiography for detection of patients with functionally significant stenosis
Authors:
Majd Zreik,
Tim Leiner,
Nadieh Khalili,
Robbert W. van Hamersvelt,
Jelmer M. Wolterink,
Michiel Voskuil,
Max A. Viergever,
Ivana Išgum
Abstract:
Treatment of patients with obstructive coronary artery disease is guided by the functional significance of a coronary artery stenosis. Fractional flow reserve (FFR), measured during invasive coronary angiography (ICA), is considered the gold standard to define the functional significance of a coronary stenosis. Here, we present a method for non-invasive detection of patients with functionally sign…
▽ More
Treatment of patients with obstructive coronary artery disease is guided by the functional significance of a coronary artery stenosis. Fractional flow reserve (FFR), measured during invasive coronary angiography (ICA), is considered the gold standard to define the functional significance of a coronary stenosis. Here, we present a method for non-invasive detection of patients with functionally significant coronary artery stenosis, combining analysis of the coronary artery tree and the left ventricular (LV) myocardium in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA scans of 126 patients who underwent invasive FFR measurements, to determine the functional significance of coronary stenoses. We combine our previous works for the analysis of the complete coronary artery tree and the LV myocardium: Coronary arteries are encoded by two disjoint convolutional autoencoders (CAEs) and the LV myocardium is characterized by a convolutional neural network (CNN) and a CAE. Thereafter, using the extracted encodings of all coronary arteries and the LV myocardium, patients are classified according to the presence of functionally significant stenosis, as defined by the invasively measured FFR. To handle the varying number of coronary arteries in a patient, the classification is formulated as a multiple instance learning problem and is performed using an attention-based neural network. Cross-validation experiments resulted in an average area under the receiver operating characteristic curve of $0.74 \pm 0.01$, and showed that the proposed combined analysis outperformed the analysis of the coronary arteries or the LV myocardium only. The results demonstrate the feasibility of combining the analyses of the complete coronary artery tree and the LV myocardium in CCTA images for the detection of patients with functionally significant stenosis in coronary arteries.
△ Less
Submitted 10 November, 2019;
originally announced November 2019.
-
Graph Convolutional Networks for Coronary Artery Segmentation in Cardiac CT Angiography
Authors:
Jelmer M. Wolterink,
Tim Leiner,
Ivana Išgum
Abstract:
Detection of coronary artery stenosis in coronary CT angiography (CCTA) requires highly personalized surface meshes enclosing the coronary lumen. In this work, we propose to use graph convolutional networks (GCNs) to predict the spatial location of vertices in a tubular surface mesh that segments the coronary artery lumen. Predictions for individual vertex locations are based on local image featur…
▽ More
Detection of coronary artery stenosis in coronary CT angiography (CCTA) requires highly personalized surface meshes enclosing the coronary lumen. In this work, we propose to use graph convolutional networks (GCNs) to predict the spatial location of vertices in a tubular surface mesh that segments the coronary artery lumen. Predictions for individual vertex locations are based on local image features as well as on features of neighboring vertices in the mesh graph. The method was trained and evaluated using the publicly available Coronary Artery Stenoses Detection and Quantification Evaluation Framework. Surface meshes enclosing the full coronary artery tree were automatically extracted. A quantitative evaluation on 78 coronary artery segments showed that these meshes corresponded closely to reference annotations, with a Dice similarity coefficient of 0.75/0.73, a mean surface distance of 0.25/0.28 mm, and a Hausdorff distance of 1.53/1.86 mm in healthy/diseased vessel segments. The results showed that inclusion of mesh information in a GCN improves segmentation overlap and accuracy over a baseline model without interaction on the mesh. The results indicate that GCNs allow efficient extraction of coronary artery surface meshes and that the use of GCNs leads to regular and more accurate meshes.
△ Less
Submitted 14 August, 2019;
originally announced August 2019.
-
Deep learning analysis of coronary arteries in cardiac CT angiography for detection of patients requiring invasive coronary angiography
Authors:
Majd Zreik,
Robbert W. van Hamersvelt,
Nadieh Khalili,
Jelmer M. Wolterink,
Michiel Voskuil,
Max A. Viergever,
Tim Leiner,
Ivana Išgum
Abstract:
In patients with obstructive coronary artery disease, the functional significance of a coronary artery stenosis needs to be determined to guide treatment. This is typically established through fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA). We present a method for automatic and non-invasive detection of patients requiring ICA, employing deep unsuper…
▽ More
In patients with obstructive coronary artery disease, the functional significance of a coronary artery stenosis needs to be determined to guide treatment. This is typically established through fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA). We present a method for automatic and non-invasive detection of patients requiring ICA, employing deep unsupervised analysis of complete coronary arteries in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA scans of 187 patients, 137 of them underwent invasive FFR measurement in 192 different coronary arteries. These FFR measurements served as a reference standard for the functional significance of the coronary stenosis. The centerlines of the coronary arteries were extracted and used to reconstruct straightened multi-planar reformatted (MPR) volumes. To automatically identify arteries with functionally significant stenosis that require ICA, each MPR volume was encoded into a fixed number of encodings using two disjoint 3D and 1D convolutional autoencoders performing spatial and sequential encodings, respectively. Thereafter, these encodings were employed to classify arteries using a support vector machine classifier. The detection of coronary arteries requiring invasive evaluation, evaluated using repeated cross-validation experiments, resulted in an area under the receiver operating characteristic curve of $0.81 \pm 0.02$ on the artery-level, and $0.87 \pm 0.02$ on the patient-level. The results demonstrate the feasibility of automatic non-invasive detection of patients that require ICA and possibly subsequent coronary artery intervention. This could potentially reduce the number of patients that unnecessarily undergo ICA.
△ Less
Submitted 10 November, 2019; v1 submitted 11 June, 2019;
originally announced June 2019.
-
Direct Automatic Coronary Calcium Scoring in Cardiac and Chest CT
Authors:
Bob D. de Vos,
Jelmer M. Wolterink,
Tim Leiner,
Pim A. de Jong,
Nikolas Lessmann,
Ivana Isgum
Abstract:
Cardiovascular disease (CVD) is the global leading cause of death. A strong risk factor for CVD events is the amount of coronary artery calcium (CAC). To meet demands of the increasing interest in quantification of CAC, i.e. coronary calcium scoring, especially as an unrequested finding for screening and research, automatic methods have been proposed. Current automatic calcium scoring methods are…
▽ More
Cardiovascular disease (CVD) is the global leading cause of death. A strong risk factor for CVD events is the amount of coronary artery calcium (CAC). To meet demands of the increasing interest in quantification of CAC, i.e. coronary calcium scoring, especially as an unrequested finding for screening and research, automatic methods have been proposed. Current automatic calcium scoring methods are relatively computationally expensive and only provide scores for one type of CT. To address this, we propose a computationally efficient method that employs two ConvNets: the first performs registration to align the fields of view of input CTs and the second performs direct regression of the calcium score, thereby circumventing time-consuming intermediate CAC segmentation. Optional decision feedback provides insight in the regions that contributed to the calcium score. Experiments were performed using 903 cardiac CT and 1,687 chest CT scans. The method predicted calcium scores in less than 0.3 s. Intra-class correlation coefficient between predicted and manual calcium scores was 0.98 for both cardiac and chest CT. The method showed almost perfect agreement between automatic and manual CVD risk categorization in both datasets, with a linearly weighted Cohen's kappa of 0.95 in cardiac CT and 0.93 in chest CT. Performance is similar to that of state-of-the-art methods, but the proposed method is hundreds of times faster. By providing visual feedback, insight is given in the decision process, making it readily implementable in clinical and research settings.
△ Less
Submitted 12 February, 2019;
originally announced February 2019.
-
Improving Myocardium Segmentation in Cardiac CT Angiography using Spectral Information
Authors:
Steffen Bruns,
Jelmer M. Wolterink,
Robbert W. van Hamersvelt,
Majd Zreik,
Tim Leiner,
Ivana Išgum
Abstract:
Accurate segmentation of the left ventricle myocardium in cardiac CT angiography (CCTA) is essential for e.g. the assessment of myocardial perfusion. Automatic deep learning methods for segmentation in CCTA might suffer from differences in contrast-agent attenuation between training and test data due to non-standardized contrast administration protocols and varying cardiac output. We propose augme…
▽ More
Accurate segmentation of the left ventricle myocardium in cardiac CT angiography (CCTA) is essential for e.g. the assessment of myocardial perfusion. Automatic deep learning methods for segmentation in CCTA might suffer from differences in contrast-agent attenuation between training and test data due to non-standardized contrast administration protocols and varying cardiac output. We propose augmentation of the training data with virtual mono-energetic reconstructions from a spectral CT scanner which show different attenuation levels of the contrast agent. We compare this to an augmentation by linear scaling of all intensity values, and combine both types of augmentation. We train a 3D fully convolutional network (FCN) with 10 conventional CCTA images and corresponding virtual mono-energetic reconstructions acquired on a spectral CT scanner, and evaluate on 40 CCTA scans acquired on a conventional CT scanner. We show that training with data augmentation using virtual mono-energetic images improves upon training with only conventional images (Dice similarity coefficient (DSC) 0.895 $\pm$ 0.039 vs. 0.846 $\pm$ 0.125). In comparison, training with data augmentation using linear scaling improves the DSC to 0.890 $\pm$ 0.039. Moreover, combining the results of both augmentation methods leads to a DSC of 0.901 $\pm$ 0.036, showing that both augmentations lead to different local improvements of the segmentations. Our results indicate that virtual mono-energetic images improve the generalization of an FCN used for myocardium segmentation in CCTA images.
△ Less
Submitted 28 January, 2019; v1 submitted 27 September, 2018;
originally announced October 2018.
-
Coronary Artery Centerline Extraction in Cardiac CT Angiography Using a CNN-Based Orientation Classifier
Authors:
Jelmer M. Wolterink,
Robbert W. van Hamersvelt,
Max A. Viergever,
Tim Leiner,
Ivana Išgum
Abstract:
Coronary artery centerline extraction in cardiac CT angiography (CCTA) images is a prerequisite for evaluation of stenoses and atherosclerotic plaque. We propose an algorithm that extracts coronary artery centerlines in CCTA using a convolutional neural network (CNN).
A 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based…
▽ More
Coronary artery centerline extraction in cardiac CT angiography (CCTA) images is a prerequisite for evaluation of stenoses and atherosclerotic plaque. We propose an algorithm that extracts coronary artery centerlines in CCTA using a convolutional neural network (CNN).
A 3D dilated CNN is trained to predict the most likely direction and radius of an artery at any given point in a CCTA image based on a local image patch. Starting from a single seed point placed manually or automatically anywhere in a coronary artery, a tracker follows the vessel centerline in two directions using the predictions of the CNN. Tracking is terminated when no direction can be identified with high certainty.
The CNN was trained using 32 manually annotated centerlines in a training set consisting of 8 CCTA images provided in the MICCAI 2008 Coronary Artery Tracking Challenge (CAT08). Evaluation using 24 test images of the CAT08 challenge showed that extracted centerlines had an average overlap of 93.7% with 96 manually annotated reference centerlines. Extracted centerline points were highly accurate, with an average distance of 0.21 mm to reference centerline points. In a second test set consisting of 50 CCTA scans, 5,448 markers in the coronary arteries were used as seed points to extract single centerlines. This showed strong correspondence between extracted centerlines and manually placed markers. In a third test set containing 36 CCTA scans, fully automatic seeding and centerline extraction led to extraction of on average 92% of clinically relevant coronary artery segments.
The proposed method is able to accurately and efficiently determine the direction and radius of coronary arteries. The method can be trained with limited training data, and once trained allows fast automatic or interactive extraction of coronary artery trees from CCTA images.
△ Less
Submitted 24 October, 2018; v1 submitted 7 October, 2018;
originally announced October 2018.
-
CNN-based Landmark Detection in Cardiac CTA Scans
Authors:
Julia M. H. Noothout,
Bob D. de Vos,
Jelmer M. Wolterink,
Tim Leiner,
Ivana Išgum
Abstract:
Fast and accurate anatomical landmark detection can benefit many medical image analysis methods. Here, we propose a method to automatically detect anatomical landmarks in medical images. Automatic landmark detection is performed with a patch-based fully convolutional neural network (FCNN) that combines regression and classification. For any given image patch, regression is used to predict the 3D d…
▽ More
Fast and accurate anatomical landmark detection can benefit many medical image analysis methods. Here, we propose a method to automatically detect anatomical landmarks in medical images. Automatic landmark detection is performed with a patch-based fully convolutional neural network (FCNN) that combines regression and classification. For any given image patch, regression is used to predict the 3D displacement vector from the image patch to the landmark. Simultaneously, classification is used to identify patches that contain the landmark. Under the assumption that patches close to a landmark can determine the landmark location more precisely than patches farther from it, only those patches that contain the landmark according to classification are used to determine the landmark location. The landmark location is obtained by calculating the average landmark location using the computed 3D displacement vectors. The method is evaluated using detection of six clinically relevant landmarks in coronary CT angiography (CCTA) scans: the right and left ostium, the bifurcation of the left main coronary artery (LM) into the left anterior descending and the left circumflex artery, and the origin of the right, non-coronary, and left aortic valve commissure. The proposed method achieved an average Euclidean distance error of 2.19 mm and 2.88 mm for the right and left ostium respectively, 3.78 mm for the bifurcation of the LM, and 1.82 mm, 2.10 mm and 1.89 mm for the origin of the right, non-coronary, and left aortic valve commissure respectively, demonstrating accurate performance. The proposed combination of regression and classification can be used to accurately detect landmarks in CCTA scans.
△ Less
Submitted 13 April, 2018;
originally announced April 2018.
-
Blood Vessel Geometry Synthesis using Generative Adversarial Networks
Authors:
Jelmer M. Wolterink,
Tim Leiner,
Ivana Isgum
Abstract:
Computationally synthesized blood vessels can be used for training and evaluation of medical image analysis applications. We propose a deep generative model to synthesize blood vessel geometries, with an application to coronary arteries in cardiac CT angiography (CCTA).
In the proposed method, a Wasserstein generative adversarial network (GAN) consisting of a generator and a discriminator networ…
▽ More
Computationally synthesized blood vessels can be used for training and evaluation of medical image analysis applications. We propose a deep generative model to synthesize blood vessel geometries, with an application to coronary arteries in cardiac CT angiography (CCTA).
In the proposed method, a Wasserstein generative adversarial network (GAN) consisting of a generator and a discriminator network is trained. While the generator tries to synthesize realistic blood vessel geometries, the discriminator tries to distinguish synthesized geometries from those of real blood vessels. Both real and synthesized blood vessel geometries are parametrized as 1D signals based on the central vessel axis. The generator can optionally be provided with an attribute vector to synthesize vessels with particular characteristics.
The GAN was optimized using a reference database with parametrizations of 4,412 real coronary artery geometries extracted from CCTA scans. After training, plausible coronary artery geometries could be synthesized based on random vectors sampled from a latent space. A qualitative analysis showed strong similarities between real and synthesized coronary arteries. A detailed analysis of the latent space showed that the diversity present in coronary artery anatomy was accurately captured by the generator.
Results show that Wasserstein generative adversarial networks can be used to synthesize blood vessel geometries.
△ Less
Submitted 12 April, 2018;
originally announced April 2018.
-
A Recurrent CNN for Automatic Detection and Classification of Coronary Artery Plaque and Stenosis in Coronary CT Angiography
Authors:
Majd Zreik,
Robbert W. van Hamersvelt,
Jelmer M. Wolterink,
Tim Leiner,
Max A. Viergever,
Ivana Isgum
Abstract:
Various types of atherosclerotic plaque and varying grades of stenosis could lead to different management of patients with coronary artery disease. Therefore, it is crucial to detect and classify the type of coronary artery plaque, as well as to detect and determine the degree of coronary artery stenosis. This study includes retrospectively collected clinically obtained coronary CT angiography (CC…
▽ More
Various types of atherosclerotic plaque and varying grades of stenosis could lead to different management of patients with coronary artery disease. Therefore, it is crucial to detect and classify the type of coronary artery plaque, as well as to detect and determine the degree of coronary artery stenosis. This study includes retrospectively collected clinically obtained coronary CT angiography (CCTA) scans of 163 patients. To perform automatic analysis for coronary artery plaque and stenosis classification, a multi-task recurrent convolutional neural network is applied on multi-planar reformatted (MPR) images of the coronary arteries. First, a 3D convolutional neural network is utilized to extract features along the coronary artery. Subsequently, the extracted features are aggregated by a recurrent neural network that performs two simultaneous multi-class classification tasks. In the first task, the network detects and characterizes the type of the coronary artery plaque (no plaque, non-calcified, mixed, calcified). In the second task, the network detects and determines the anatomical significance of the coronary artery stenosis (no stenosis, non-significant i.e. <50% luminal narrowing, significant i.e. >50% luminal narrowing). For detection and classification of coronary plaque, the method achieved an accuracy of 0.77. For detection and classification of stenosis, the method achieved an accuracy of 0.80. The results demonstrate that automatic detection and classification of coronary artery plaque and stenosis are feasible. This may enable automated triage of patients to those without coronary plaque and those with coronary plaque and stenosis in need for further cardiovascular workup.
△ Less
Submitted 10 December, 2018; v1 submitted 12 April, 2018;
originally announced April 2018.
-
Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis
Authors:
Majd Zreik,
Nikolas Lessmann,
Robbert W. van Hamersvelt,
Jelmer M. Wolterink,
Michiel Voskuil,
Max A. Viergever,
Tim Leiner,
Ivana Išgum
Abstract:
In patients with coronary artery stenoses of intermediate severity, the functional significance needs to be determined. Fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA), is most often used in clinical practice. To reduce the number of ICA procedures, we present a method for automatic identification of patients with functionally significant coronary ar…
▽ More
In patients with coronary artery stenoses of intermediate severity, the functional significance needs to be determined. Fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA), is most often used in clinical practice. To reduce the number of ICA procedures, we present a method for automatic identification of patients with functionally significant coronary artery stenoses, employing deep learning analysis of the left ventricle (LV) myocardium in rest coronary CT angiography (CCTA). The study includes consecutively acquired CCTA scans of 166 patients with FFR measurements. To identify patients with a functionally significant coronary artery stenosis, analysis is performed in several stages. First, the LV myocardium is segmented using a multiscale convolutional neural network (CNN). To characterize the segmented LV myocardium, it is subsequently encoded using unsupervised convolutional autoencoder (CAE). Thereafter, patients are classified according to the presence of functionally significant stenosis using an SVM classifier based on the extracted and clustered encodings. Quantitative evaluation of LV myocardium segmentation in 20 images resulted in an average Dice coefficient of 0.91 and an average mean absolute distance between the segmented and reference LV boundaries of 0.7 mm. Classification of patients was evaluated in the remaining 126 CCTA scans in 50 10-fold cross-validation experiments and resulted in an area under the receiver operating characteristic curve of 0.74 +- 0.02. At sensitivity levels 0.60, 0.70 and 0.80, the corresponding specificity was 0.77, 0.71 and 0.59, respectively. The results demonstrate that automatic analysis of the LV myocardium in a single CCTA scan acquired at rest, without assessment of the anatomy of the coronary arteries, can be used to identify patients with functionally significant coronary artery stenosis.
△ Less
Submitted 6 December, 2017; v1 submitted 24 November, 2017;
originally announced November 2017.
-
Automatic Segmentation and Disease Classification Using Cardiac Cine MR Images
Authors:
Jelmer M. Wolterink,
Tim Leiner,
Max A. Viergever,
Ivana Isgum
Abstract:
Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features…
▽ More
Segmentation of the heart in cardiac cine MR is clinically used to quantify cardiac function. We propose a fully automatic method for segmentation and disease classification using cardiac cine MR images. A convolutional neural network (CNN) was designed to simultaneously segment the left ventricle (LV), right ventricle (RV) and myocardium in end-diastole (ED) and end-systole (ES) images. Features derived from the obtained segmentations were used in a Random Forest classifier to label patients as suffering from dilated cardiomyopathy, hypertrophic cardiomyopathy, heart failure following myocardial infarction, right ventricular abnormality, or no cardiac disease. The method was developed and evaluated using a balanced dataset containing images of 100 patients, which was provided in the MICCAI 2017 automated cardiac diagnosis challenge (ACDC). The segmentation and classification pipeline were evaluated in a four-fold stratified cross-validation. Average Dice scores between reference and automatically obtained segmentations were 0.94, 0.88 and 0.87 for the LV, RV and myocardium. The classifier assigned 91% of patients to the correct disease category. Segmentation and disease classification took 5 s per patient. The results of our study suggest that image-based diagnosis using cine MR cardiac scans can be performed automatically with high accuracy.
△ Less
Submitted 3 August, 2017;
originally announced August 2017.
-
Automatic Segmentation of the Left Ventricle in Cardiac CT Angiography Using Convolutional Neural Network
Authors:
Majd Zreik,
Tim Leiner,
Bob D. de Vos,
Robbert W. van Hamersvelt,
Max A. Viergever,
Ivana Isgum
Abstract:
Accurate delineation of the left ventricle (LV) is an important step in evaluation of cardiac function. In this paper, we present an automatic method for segmentation of the LV in cardiac CT angiography (CCTA) scans. Segmentation is performed in two stages. First, a bounding box around the LV is detected using a combination of three convolutional neural networks (CNNs). Subsequently, to obtain the…
▽ More
Accurate delineation of the left ventricle (LV) is an important step in evaluation of cardiac function. In this paper, we present an automatic method for segmentation of the LV in cardiac CT angiography (CCTA) scans. Segmentation is performed in two stages. First, a bounding box around the LV is detected using a combination of three convolutional neural networks (CNNs). Subsequently, to obtain the segmentation of the LV, voxel classification is performed within the defined bounding box using a CNN. The study included CCTA scans of sixty patients, fifty scans were used to train the CNNs for the LV localization, five scans were used to train LV segmentation and the remaining five scans were used for testing the method. Automatic segmentation resulted in the average Dice coefficient of 0.85 and mean absolute surface distance of 1.1 mm. The results demonstrate that automatic segmentation of the LV in CCTA scans using voxel classification with convolutional neural networks is feasible.
△ Less
Submitted 19 April, 2017;
originally announced April 2017.
-
ConvNet-Based Localization of Anatomical Structures in 3D Medical Images
Authors:
Bob D. de Vos,
Jelmer M. Wolterink,
Pim A. de Jong,
Tim Leiner,
Max A. Viergever,
Ivana Išgum
Abstract:
Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet).
A single ConvNet is trained to detect presence of the anatomical structure of interest in ax…
▽ More
Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet).
A single ConvNet is trained to detect presence of the anatomical structure of interest in axial, coronal, and sagittal slices extracted from a 3D image. To allow the ConvNet to analyze slices of different sizes, spatial pyramid pooling is applied. After detection, 3D bounding boxes are created by combining the output of the ConvNet in all slices.
In the experiments 200 chest CT, 100 cardiac CT angiography (CTA), and 100 abdomen CT scans were used. The heart, ascending aorta, aortic arch, and descending aorta were localized in chest CT scans, the left cardiac ventricle in cardiac CTA scans, and the liver in abdomen CT scans. Localization was evaluated using the distances between automatically and manually defined reference bounding box centroids and walls.
The best results were achieved in localization of structures with clearly defined boundaries (e.g. aortic arch) and the worst when the structure boundary was not clearly visible (e.g. liver). The method was more robust and accurate in localization multiple structures.
△ Less
Submitted 19 April, 2017;
originally announced April 2017.
-
Dilated Convolutional Neural Networks for Cardiovascular MR Segmentation in Congenital Heart Disease
Authors:
Jelmer M. Wolterink,
Tim Leiner,
Max A. Viergever,
Ivana Išgum
Abstract:
We propose an automatic method using dilated convolutional neural networks (CNNs) for segmentation of the myocardium and blood pool in cardiovascular MR (CMR) of patients with congenital heart disease (CHD).
Ten training and ten test CMR scans cropped to an ROI around the heart were provided in the MICCAI 2016 HVSMR challenge. A dilated CNN with a receptive field of 131x131 voxels was trained fo…
▽ More
We propose an automatic method using dilated convolutional neural networks (CNNs) for segmentation of the myocardium and blood pool in cardiovascular MR (CMR) of patients with congenital heart disease (CHD).
Ten training and ten test CMR scans cropped to an ROI around the heart were provided in the MICCAI 2016 HVSMR challenge. A dilated CNN with a receptive field of 131x131 voxels was trained for myocardium and blood pool segmentation in axial, sagittal and coronal image slices. Performance was evaluated within the HVSMR challenge.
Automatic segmentation of the test scans resulted in Dice indices of 0.80$\pm$0.06 and 0.93$\pm$0.02, average distances to boundaries of 0.96$\pm$0.31 and 0.89$\pm$0.24 mm, and Hausdorff distances of 6.13$\pm$3.76 and 7.07$\pm$3.01 mm for the myocardium and blood pool, respectively. Segmentation took 41.5$\pm$14.7 s per scan.
In conclusion, dilated CNNs trained on a small set of CMR images of CHD patients showing large anatomical variability provide accurate myocardium and blood pool segmentations.
△ Less
Submitted 12 April, 2017;
originally announced April 2017.
-
Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities
Authors:
Pim Moeskops,
Jelmer M. Wolterink,
Bas H. M. van der Velden,
Kenneth G. A. Gilhuijs,
Tim Leiner,
Max A. Viergever,
Ivana Išgum
Abstract:
Automatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single convolutional neural network (CNN) can be trained to perform different segmentation tasks.
A single CNN is trained to segment six tissues in MR brain image…
▽ More
Automatic segmentation of medical images is an important task for many clinical applications. In practice, a wide range of anatomical structures are visualised using different imaging modalities. In this paper, we investigate whether a single convolutional neural network (CNN) can be trained to perform different segmentation tasks.
A single CNN is trained to segment six tissues in MR brain images, the pectoral muscle in MR breast images, and the coronary arteries in cardiac CTA. The CNN therefore learns to identify the imaging modality, the visualised anatomical structures, and the tissue classes.
For each of the three tasks (brain MRI, breast MRI and cardiac CTA), this combined training procedure resulted in a segmentation performance equivalent to that of a CNN trained specifically for that task, demonstrating the high capacity of CNN architectures. Hence, a single system could be used in clinical practice to automatically perform diverse segmentation tasks without task-specific training.
△ Less
Submitted 11 April, 2017;
originally announced April 2017.
