-
Automated Quality Control in Image Segmentation: Application to the UK Biobank Cardiac MR Imaging Study
Authors:
Robert Robinson,
Vanya V. Valindria,
Wenjia Bai,
Ozan Oktay,
Bernhard Kainz,
Hideaki Suzuki,
Mihir M. Sanghvi,
Nay Aung,
Jos$é$ Miguel Paiva,
Filip Zemrak,
Kenneth Fung,
Elena Lukaschuk,
Aaron M. Lee,
Valentina Carapella,
Young ** Kim,
Stefan K. Piechnik,
Stefan Neubauer,
Steffen E. Petersen,
Chris Page,
Paul M. Matthews,
Daniel Rueckert,
Ben Glocker
Abstract:
Background: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools, e.g. image segmentation methods, are employed to derive quantitative measures or biomarkers for later analyses. Manual inspection and visual QC of each segmentation isn't feasible at large scale. However, i…
▽ More
Background: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools, e.g. image segmentation methods, are employed to derive quantitative measures or biomarkers for later analyses. Manual inspection and visual QC of each segmentation isn't feasible at large scale. However, it's important to be able to automatically detect when a segmentation method fails so as to avoid inclusion of wrong measurements into subsequent analyses which could lead to incorrect conclusions. Methods: To overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4,800 cardiac magnetic resonance scans. We then apply our method to a large cohort of 7,250 cardiac MRI on which we have performed manual QC. Results: We report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4,800 scans for which manual segmentations were available. We mimic real-world application of the method on 7,250 cardiac MRI where we show good agreement between predicted quality metrics and manual visual QC scores. Conclusions: We show that RCA has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study.
△ Less
Submitted 27 January, 2019;
originally announced January 2019.
-
Real-time Prediction of Segmentation Quality
Authors:
Robert Robinson,
Ozan Oktay,
Wenjia Bai,
Vanya Valindria,
Mihir Sanghvi,
Nay Aung,
José Paiva,
Filip Zemrak,
Kenneth Fung,
Elena Lukaschuk,
Aaron Lee,
Valentina Carapella,
Young ** Kim,
Bernhard Kainz,
Stefan Piechnik,
Stefan Neubauer,
Steffen Petersen,
Chris Page,
Daniel Rueckert,
Ben Glocker
Abstract:
Recent advances in deep learning based image segmentation methods have enabled real-time performance with human-level accuracy. However, occasionally even the best method fails due to low image quality, artifacts or unexpected behaviour of black box algorithms. Being able to predict segmentation quality in the absence of ground truth is of paramount importance in clinical practice, but also in lar…
▽ More
Recent advances in deep learning based image segmentation methods have enabled real-time performance with human-level accuracy. However, occasionally even the best method fails due to low image quality, artifacts or unexpected behaviour of black box algorithms. Being able to predict segmentation quality in the absence of ground truth is of paramount importance in clinical practice, but also in large-scale studies to avoid the inclusion of invalid data in subsequent analysis.
In this work, we propose two approaches of real-time automated quality control for cardiovascular MR segmentations using deep learning. First, we train a neural network on 12,880 samples to predict Dice Similarity Coefficients (DSC) on a per-case basis. We report a mean average error (MAE) of 0.03 on 1,610 test samples and 97% binary classification accuracy for separating low and high quality segmentations. Secondly, in the scenario where no manually annotated data is available, we train a network to predict DSC scores from estimated quality obtained via a reverse testing strategy. We report an MAE=0.14 and 91% binary classification accuracy for this case. Predictions are obtained in real-time which, when combined with real-time segmentation methods, enables instant feedback on whether an acquired scan is analysable while the patient is still in the scanner. This further enables new applications of optimising image acquisition towards best possible analysis results.
△ Less
Submitted 16 June, 2018;
originally announced June 2018.
-
Automated cardiovascular magnetic resonance image analysis with fully convolutional networks
Authors:
Wenjia Bai,
Matthew Sinclair,
Giacomo Tarroni,
Ozan Oktay,
Martin Rajchl,
Ghislain Vaillant,
Aaron M. Lee,
Nay Aung,
Elena Lukaschuk,
Mihir M. Sanghvi,
Filip Zemrak,
Kenneth Fung,
Jose Miguel Paiva,
Valentina Carapella,
Young ** Kim,
Hideaki Suzuki,
Bernhard Kainz,
Paul M. Matthews,
Steffen E. Petersen,
Stefan K. Piechnik,
Stefan Neubauer,
Ben Glocker,
Daniel Rueckert
Abstract:
Cardiovascular magnetic resonance (CMR) imaging is a standard imaging modality for assessing cardiovascular diseases (CVDs), the leading cause of death globally. CMR enables accurate quantification of the cardiac chamber volume, ejection fraction and myocardial mass, providing information for diagnosis and monitoring of CVDs. However, for years, clinicians have been relying on manual approaches fo…
▽ More
Cardiovascular magnetic resonance (CMR) imaging is a standard imaging modality for assessing cardiovascular diseases (CVDs), the leading cause of death globally. CMR enables accurate quantification of the cardiac chamber volume, ejection fraction and myocardial mass, providing information for diagnosis and monitoring of CVDs. However, for years, clinicians have been relying on manual approaches for CMR image analysis, which is time consuming and prone to subjective errors. It is a major clinical challenge to automatically derive quantitative and clinically relevant information from CMR images. Deep neural networks have shown a great potential in image pattern recognition and segmentation for a variety of tasks. Here we demonstrate an automated analysis method for CMR images, which is based on a fully convolutional network (FCN). The network is trained and evaluated on a large-scale dataset from the UK Biobank, consisting of 4,875 subjects with 93,500 pixelwise annotated images. The performance of the method has been evaluated using a number of technical metrics, including the Dice metric, mean contour distance and Hausdorff distance, as well as clinically relevant measures, including left ventricle (LV) end-diastolic volume (LVEDV) and end-systolic volume (LVESV), LV mass (LVM); right ventricle (RV) end-diastolic volume (RVEDV) and end-systolic volume (RVESV). By combining FCN with a large-scale annotated dataset, the proposed automated method achieves a high performance on par with human experts in segmenting the LV and RV on short-axis CMR images and the left atrium (LA) and right atrium (RA) on long-axis CMR images.
△ Less
Submitted 22 May, 2018; v1 submitted 25 October, 2017;
originally announced October 2017.
-
Mathematical Modelling of Heart Rate Changes in the Mouse
Authors:
Mark Christie,
Manasi Nandi,
Yanika Borg,
Valentina Carapella,
Gary Mirams,
Philip Aston,
Saziye Bayram,
Radostin D. Simitev,
Jennifer Siggers,
Buddhapriya Chakrabarti
Abstract:
The CVS is composed of numerous interacting and dynamically regulated physiological subsystems which each generate measurable periodic components such that the CVS can itself be presented as a system of weakly coupled oscillators. The interactions between these oscillators generate a chaotic blood pressure waveform signal, where periods of apparent rhythmicity are punctuated by asynchronous behavi…
▽ More
The CVS is composed of numerous interacting and dynamically regulated physiological subsystems which each generate measurable periodic components such that the CVS can itself be presented as a system of weakly coupled oscillators. The interactions between these oscillators generate a chaotic blood pressure waveform signal, where periods of apparent rhythmicity are punctuated by asynchronous behaviour. It is this variability which seems to characterise the normal state. We used a standard experimental data set for the purposes of analysis and modelling. Arterial blood pressure waveform data was collected from conscious mice instrumented with radiotelemetry devices over $24$ hours, at a $100$ Hz and $1$ kHz time base. During a $24$ hour period, these mice display diurnal variation leading to changes in the cardiovascular waveform. We undertook preliminary analysis of our data using Fourier transforms and subsequently applied a series of both linear and nonlinear mathematical approaches in parallel. We provide a minimalistic linear and nonlinear coupled oscillator model and employed spectral and Hilbert analysis as well as a phase plane analysis. This provides a route to a three way synergistic investigation of the original blood pressure data by a combination of physiological experiments, data analysis viz. Fourier and Hilbert transforms and attractor reconstructions, and numerical solutions of linear and nonlinear coupled oscillator models. We believe that a minimal model of coupled oscillator models that quantitatively describes the complex physiological data could be developed via such a method. Further investigations of each of these techniques will be explored in separate publications.
△ Less
Submitted 5 October, 2015;
originally announced October 2015.
