-
Inline AI: Open-source Deep Learning Inference for Cardiac MR
Authors:
Hui Xue,
Rhodri H Davies,
James Howard,
Hunain Shiwani,
Azaan Rehman,
Iain Pierce,
Henry Procter,
Marianna Fontana,
James C Moon,
Eylem Levelt,
Peter Kellman
Abstract:
Cardiac Magnetic Resonance (CMR) is established as a non-invasive imaging technique for evaluation of heart function, anatomy, and myocardial tissue characterization. Quantitative biomarkers are central for diagnosis and management of heart disease. Deep learning (DL) is playing an ever more important role in extracting these quantitative measures from CMR images. While many researchers have repor…
▽ More
Cardiac Magnetic Resonance (CMR) is established as a non-invasive imaging technique for evaluation of heart function, anatomy, and myocardial tissue characterization. Quantitative biomarkers are central for diagnosis and management of heart disease. Deep learning (DL) is playing an ever more important role in extracting these quantitative measures from CMR images. While many researchers have reported promising results in training and evaluating models, model deployment into the imaging workflow is less explored.
A new imaging AI framework, the InlineAI, was developed and open-sourced. The main innovation is to enable the model inference inline as a part of imaging computation, instead of as an offline post-processing step and to allow users to plug in their models. We demonstrate the system capability on three applications: long-axis CMR cine landmark detection, short-axis CMR cine analysis of function and anatomy, and quantitative perfusion map**.
The InlineAI allowed models to be deployed into imaging workflow in a streaming manner directly on the scanner. The model was loaded and inference on incoming images were performed while the data acquisition was ongoing, and results were sent back to scanner. Several biomarkers were extracted from model outputs in the demonstrated applications and reported as curves and tabular values. All processes are full automated. the model inference was completed within 6-45s after the end of imaging data acquisition.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
A 3D deep learning classifier and its explainability when assessing coronary artery disease
Authors:
Wing Keung Cheung,
Jeremy Kalindjian,
Robert Bell,
Arjun Nair,
Leon J. Menezes,
Riyaz Patel,
Simon Wan,
Kacy Chou,
Jiahang Chen,
Ryo Torii,
Rhodri H. Davies,
James C. Moon,
Daniel C. Alexander,
Joseph Jacob
Abstract:
Early detection and diagnosis of coronary artery disease (CAD) could save lives and reduce healthcare costs. In this study, we propose a 3D Resnet-50 deep learning model to directly classify normal subjects and CAD patients on computed tomography coronary angiography images. Our proposed method outperforms a 2D Resnet-50 model by 23.65%. Explainability is also provided by using a Grad-GAM. Further…
▽ More
Early detection and diagnosis of coronary artery disease (CAD) could save lives and reduce healthcare costs. In this study, we propose a 3D Resnet-50 deep learning model to directly classify normal subjects and CAD patients on computed tomography coronary angiography images. Our proposed method outperforms a 2D Resnet-50 model by 23.65%. Explainability is also provided by using a Grad-GAM. Furthermore, we link the 3D CAD classification to a 2D two-class semantic segmentation for improved explainability and accurate abnormality localisation.
△ Less
Submitted 29 July, 2023;
originally announced August 2023.
-
Landmark detection in Cardiac Magnetic Resonance Imaging Using A Convolutional Neural Network
Authors:
Hui Xue,
Jessica Artico,
Marianna Fontana,
James C Moon,
Rhodri H Davies,
Peter Kellman
Abstract:
Purpose: To develop a convolutional neural network (CNN) solution for robust landmark detection in cardiac MR images.
Methods: This retrospective study included cine, LGE and T1 map** scans from two hospitals. The training set included 2,329 patients and 34,019 images. A hold-out test set included 531 patients and 7,723 images. CNN models were developed to detect two mitral valve plane and api…
▽ More
Purpose: To develop a convolutional neural network (CNN) solution for robust landmark detection in cardiac MR images.
Methods: This retrospective study included cine, LGE and T1 map** scans from two hospitals. The training set included 2,329 patients and 34,019 images. A hold-out test set included 531 patients and 7,723 images. CNN models were developed to detect two mitral valve plane and apical points on long-axis (LAX) images. On short-axis (SAX) images, anterior and posterior RV insertion points and LV center were detected. Model outputs were compared to manual labels by two operators for accuracy with a t-test for statistical significance. The trained model was deployed to MR scanners.
Results: For the LAX images, success detection was 99.8% for cine, 99.4% for LGE. For the SAX, success rate was 96.6%, 97.6% and 98.9% for cine, LGE and T1-map**. The L2 distances between model and manual labels were 2 to 3.5 mm, indicating close agreement between model landmarks to manual labels. No significant differences were found for the anterior RV insertion angle and LV length by the models and operators for all views and imaging sequences. Model inference on MR scanner took 610ms/5.6s on GPU/CPU, respectively, for a typical cardiac cine series.
Conclusions: This study developed, validated and deployed a CNN solution for robust landmark detection in both long and short-axis CMR images for cine, LGE and T1 map** sequences, with the accuracy comparable to the inter-operator variation.
△ Less
Submitted 13 August, 2020;
originally announced August 2020.
-
Automated Inline Analysis of Myocardial Perfusion MRI with Deep Learning
Authors:
Hui Xue,
Rhodri Davies,
Louis AE Brown,
Kristopher D Knott,
Tushar Kotecha,
Marianna Fontana,
Sven Plein,
James C Moon,
Peter Kellman
Abstract:
Recent development of quantitative myocardial blood flow (MBF) map** allows direct evaluation of absolute myocardial perfusion, by computing pixel-wise flow maps. Clinical studies suggest quantitative evaluation would be more desirable for objectivity and efficiency. Objective assessment can be further facilitated by segmenting the myocardium and automatically generating reports following the AH…
▽ More
Recent development of quantitative myocardial blood flow (MBF) map** allows direct evaluation of absolute myocardial perfusion, by computing pixel-wise flow maps. Clinical studies suggest quantitative evaluation would be more desirable for objectivity and efficiency. Objective assessment can be further facilitated by segmenting the myocardium and automatically generating reports following the AHA model. This will free user interaction for analysis and lead to a 'one-click' solution to improve workflow. This paper proposes a deep neural network based computational workflow for inline myocardial perfusion analysis. Adenosine stress and rest perfusion scans were acquired from three hospitals. Training set included N=1,825 perfusion series from 1,034 patients. Independent test set included 200 scans from 105 patients. Data were consecutively acquired at each site. A convolution neural net (CNN) model was trained to provide segmentation for LV cavity, myocardium and right ventricular by processing incoming 2D+T perfusion Gd series. Model outputs were compared to manual ground-truth for accuracy of segmentation and flow measures derived on global and per-sector basis. The trained models were integrated onto MR scanners for effective inference. Segmentation accuracy and myocardial flow measures were compared between CNN models and manual ground-truth. The mean Dice ratio of CNN derived myocardium was 0.93 +/- 0.04. Both global flow and per-sector values showed no significant difference, compared to manual results. The AHA 16 segment model was automatically generated and reported on the MR scanner. As a result, the fully automated analysis of perfusion flow map** was achieved. This solution was integrated on the MR scanner, enabling 'one-click' analysis and reporting of myocardial blood flow.
△ Less
Submitted 29 May, 2020; v1 submitted 1 November, 2019;
originally announced November 2019.
-
Automated Detection of Left Ventricle in Arterial Input Function Images for Inline Perfusion Map** using Deep Learning: A study of 15,000 Patients
Authors:
Hui Xue,
Ethan Tseng,
Kristopher D Knott,
Tushar Kotecha,
Louise Brown,
Sven Plein,
Marianna Fontana,
James C Moon,
Peter Kellman
Abstract:
Quantification of myocardial perfusion has the potential to improve detection of regional and global flow reduction. Significant effort has been made to automate the workflow, where one essential step is the arterial input function (AIF) extraction. Since failure here invalidates quantification, high accuracy is required. For this purpose, this study presents a robust AIF detection method using th…
▽ More
Quantification of myocardial perfusion has the potential to improve detection of regional and global flow reduction. Significant effort has been made to automate the workflow, where one essential step is the arterial input function (AIF) extraction. Since failure here invalidates quantification, high accuracy is required. For this purpose, this study presents a robust AIF detection method using the convolutional neural net (CNN) model. CNN models were trained by assembling 25,027 scans (N=12,984 patients) from three hospitals, seven scanners. A test set of 5,721 scans (N=2,805 patients) evaluated model performance. The 2D+T AIF time series was inputted into CNN. Two variations were investigated: a) Two Classes (2CS) for background and foreground (LV mask); b) Three Classes (3CS) for background, foreground LV and RV. Final model was deployed on MR scanners via the Gadgetron InlineAI. Model loading time on MR scanner was ~340ms and applying it took ~180ms. The 3CS model successfully detect LV for 99.98% of all test cases (1 failed out of 5,721 cases). The mean Dice ratio for 3CS was 0.87+/-0.08 with 92.0% of all test cases having Dice ratio >0.75, while the 2CS model gave lower Dice of 0.82+/-0.22 (P<1e-5). Extracted AIF signals using CNN were further compared to manual ground-truth for foot-time, peak-time, first-pass duration, peak value and area-under-curve. No significant differences were found for all features (P>0.2). This study proposed, validated, and deployed a robust CNN solution to detect the LV for the extraction of the AIF signal used in fully automated perfusion flow map**. A very large data cohort was assembled and resulting models were deployed to MR scanners for fully inline AI in clinical hospitals.
△ Less
Submitted 6 April, 2020; v1 submitted 15 October, 2019;
originally announced October 2019.
-
Improving the generalizability of convolutional neural network-based segmentation on CMR images
Authors:
Chen Chen,
Wenjia Bai,
Rhodri H. Davies,
Anish N. Bhuva,
Charlotte Manisty,
James C. Moon,
Nay Aung,
Aaron M. Lee,
Mihir M. Sanghvi,
Kenneth Fung,
Jose Miguel Paiva,
Steffen E. Petersen,
Elena Lukaschuk,
Stefan K. Piechnik,
Stefan Neubauer,
Daniel Rueckert
Abstract:
Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g. same scanner or site), their performance often degrades dramatically on ima…
▽ More
Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g. same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strategies to accommodate common scenarios in multi-site, multi-scanner clinical imaging data sets. We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy. Specifically, the method was trained on a large set of 3,975 subjects from the UK Biobank. It was then directly tested on 600 different subjects from the UK Biobank for intra-domain testing and two other sets for cross-domain testing: the ACDC dataset (100 subjects, 1 site, 2 scanners) and the BSCMR-AS dataset (599 subjects, 6 sites, 9 scanners). The proposed method produces promising segmentation results on the UK Biobank test set which are comparable to previously reported values in the literature, while also performing well on cross-domain test sets, achieving a mean Dice metric of 0.90 for the left ventricle, 0.81 for the myocardium and 0.82 for the right ventricle on the ACDC dataset; and 0.89 for the left ventricle, 0.83 for the myocardium on the BSCMR-AS dataset. The proposed method offers a potential solution to improve CNN-based model generalizability for the cross-scanner and cross-site cardiac MR image segmentation task.
△ Less
Submitted 3 July, 2019; v1 submitted 2 July, 2019;
originally announced July 2019.