-
Artificial intelligence for abnormality detection in high volume neuroimaging: a systematic review and meta-analysis
Authors:
Siddharth Agarwal,
David A. Wood,
Mariusz Grzeda,
Chandhini Suresh,
Munaib Din,
James Cole,
Marc Modat,
Thomas C Booth
Abstract:
Purpose: Most studies evaluating artificial intelligence (AI) models that detect abnormalities in neuroimaging are either tested on unrepresentative patient cohorts or are insufficiently well-validated, leading to poor generalisability to real-world tasks. The aim was to determine the diagnostic test accuracy and summarise the evidence supporting the use of AI models performing first-line, high-vo…
▽ More
Purpose: Most studies evaluating artificial intelligence (AI) models that detect abnormalities in neuroimaging are either tested on unrepresentative patient cohorts or are insufficiently well-validated, leading to poor generalisability to real-world tasks. The aim was to determine the diagnostic test accuracy and summarise the evidence supporting the use of AI models performing first-line, high-volume neuroimaging tasks.
Methods: Medline, Embase, Cochrane library and Web of Science were searched until September 2021 for studies that temporally or externally validated AI capable of detecting abnormalities in first-line CT or MR neuroimaging. A bivariate random-effects model was used for meta-analysis where appropriate. PROSPERO: CRD42021269563.
Results: Only 16 studies were eligible for inclusion. Included studies were not compromised by unrepresentative datasets or inadequate validation methodology. Direct comparison with radiologists was available in 4/16 studies. 15/16 had a high risk of bias. Meta-analysis was only suitable for intracranial haemorrhage detection in CT imaging (10/16 studies), where AI systems had a pooled sensitivity and specificity 0.90 (95% CI 0.85 - 0.94) and 0.90 (95% CI 0.83 - 0.95) respectively. Other AI studies using CT and MRI detected target conditions other than haemorrhage (2/16), or multiple target conditions (4/16). Only 3/16 studies implemented AI in clinical pathways, either for pre-read triage or as post-read discrepancy identifiers.
Conclusion: The paucity of eligible studies reflects that most abnormality detection AI studies were not adequately validated in representative clinical cohorts. The few studies describing how abnormality detection AI could impact patients and clinicians did not explore the full ramifications of clinical implementation.
△ Less
Submitted 9 May, 2024;
originally announced May 2024.
-
NEURO HAND: A weakly supervised Hierarchical Attention Network for interpretable neuroimaging abnormality Detection
Authors:
David A. Wood
Abstract:
Clinical neuroimaging data is naturally hierarchical. Different magnetic resonance imaging (MRI) sequences within a series, different slices covering the head, and different regions within each slice all confer different information. In this work we present a hierarchical attention network for abnormality detection using MRI scans obtained in a clinical hospital setting. The proposed network is su…
▽ More
Clinical neuroimaging data is naturally hierarchical. Different magnetic resonance imaging (MRI) sequences within a series, different slices covering the head, and different regions within each slice all confer different information. In this work we present a hierarchical attention network for abnormality detection using MRI scans obtained in a clinical hospital setting. The proposed network is suitable for non-volumetric data (i.e. stacks of high-resolution MRI slices), and can be trained from binary examination-level labels. We show that this hierarchical approach leads to improved classification, while providing interpretability through either coarse inter- and intra-slice abnormality localisation, or giving importance scores for different slices and sequences, making our model suitable for use as an automated triaging system in radiology departments.
△ Less
Submitted 16 January, 2024; v1 submitted 6 November, 2023;
originally announced November 2023.
-
Auction algorithm sensitivity for multi-robot task allocation
Authors:
Katie Clinch,
Tony A. Wood,
Chris Manzie
Abstract:
We consider the problem of finding a low-cost allocation and ordering of tasks between a team of robots in a d-dimensional, uncertain, landscape, and the sensitivity of this solution to changes in the cost function. Various algorithms have been shown to give a 2-approximation to the MinSum allocation problem. By analysing such an auction algorithm, we obtain intervals on each cost, such that any f…
▽ More
We consider the problem of finding a low-cost allocation and ordering of tasks between a team of robots in a d-dimensional, uncertain, landscape, and the sensitivity of this solution to changes in the cost function. Various algorithms have been shown to give a 2-approximation to the MinSum allocation problem. By analysing such an auction algorithm, we obtain intervals on each cost, such that any fluctuation of the costs within these intervals will result in the auction algorithm outputting the same solution.
△ Less
Submitted 28 June, 2023;
originally announced June 2023.
-
Certification of Bottleneck Task Assignment with Shortest Path Criteria
Authors:
Tony A. Wood,
Maryam Kamgarpour
Abstract:
Minimising the longest travel distance for a group of mobile robots with interchangeable goals requires knowledge of the shortest length paths between all robots and goal destinations. Determining the exact length of the shortest paths in an environment with obstacles is NP-hard however. In this paper, we investigate when polynomial-time approximations of the shortest path search are sufficient to…
▽ More
Minimising the longest travel distance for a group of mobile robots with interchangeable goals requires knowledge of the shortest length paths between all robots and goal destinations. Determining the exact length of the shortest paths in an environment with obstacles is NP-hard however. In this paper, we investigate when polynomial-time approximations of the shortest path search are sufficient to determine the optimal assignment of robots to goals. In particular, we propose an algorithm in which the accuracy of the path planning is iteratively increased. The approach provides a certificate when the uncertainties on estimates of the shortest paths become small enough to guarantee the optimality of the goal assignment. To this end, we apply results from assignment sensitivity assuming upper and lower bounds on the length of the shortest paths. We then provide polynomial-time methods to find such bounds by applying sampling-based path planning. The upper bounds are given by feasible paths, the lower bounds are obtained by expanding the sample set and leveraging the knowledge of the sample dispersion. We demonstrate the application of the proposed method with a multi-robot path-planning case study.
△ Less
Submitted 8 June, 2023; v1 submitted 24 December, 2022;
originally announced December 2022.
-
Probabilistic Data Association for Semantic SLAM at Scale
Authors:
Elad Michael,
Tyler Summers,
Tony A. Wood,
Chris Manzie,
Iman Shames
Abstract:
With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and map** (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve th…
▽ More
With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and map** (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve the ability to recognise previously visited locations, as well as maintain sparser maps for long term SLAM applications. However, SLAM in repetitive environments has the critical problem of assigning measurements to the landmarks which generated them. In this paper, we use k-best assignment enumeration to compute marginal assignment probabilities for each measurement landmark pair, in real time. We present numerical studies on the KITTI dataset to demonstrate the effectiveness and speed of the proposed framework.
△ Less
Submitted 25 February, 2022;
originally announced February 2022.
-
Temporal Logic Planning for Minimum-Time Positioning of Multiple Threat-Seduction Decoys
Authors:
Tony A. Wood,
Mitchell Khoo,
Elad Michael,
Chris Manzie,
Iman Shames
Abstract:
Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling m…
▽ More
Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling multiple decoys to minimise the time required to break the locks of all the threats. The method includes the optimal allocation of one decoy to every threat with an assignment procedure that provides local position constraints to guarantee collision avoidance and thereby decouples the control of the decoys. A crude model of a decoy with uncertainty is considered for motion planning. The task of a decoy reaching a state in which the lock of the assigned threat can be broken is formulated as a temporal logic specification. To this end, the requirements to complete the task are modelled as time-varying set-membership constraints. The temporal and logical combination of the constraints is encoded in a mixed-integer optimisation problem. To demonstrate the results a simulated case study is provided.
△ Less
Submitted 17 June, 2021;
originally announced June 2021.
-
Automated triaging of head MRI examinations using convolutional neural networks
Authors:
David A. Wood,
Sina Kafiabadi,
Ayisha Al Busaidi,
Emily Guilhem,
Antanas Montvila,
Siddharth Agarwal,
Jeremy Lynch,
Matthew Townend,
Gareth Barker,
Sebastien Ourselin,
James H. Cole,
Thomas C. Booth
Abstract:
The growing demand for head magnetic resonance imaging (MRI) examinations, along with a global shortage of radiologists, has led to an increase in the time taken to report head MRI scans around the world. For many neurological conditions, this delay can result in increased morbidity and mortality. An automated triaging tool could reduce reporting times for abnormal examinations by identifying abno…
▽ More
The growing demand for head magnetic resonance imaging (MRI) examinations, along with a global shortage of radiologists, has led to an increase in the time taken to report head MRI scans around the world. For many neurological conditions, this delay can result in increased morbidity and mortality. An automated triaging tool could reduce reporting times for abnormal examinations by identifying abnormalities at the time of imaging and prioritizing the reporting of these scans. In this work, we present a convolutional neural network for detecting clinically-relevant abnormalities in $\text{T}_2$-weighted head MRI scans. Using a validated neuroradiology report classifier, we generated a labelled dataset of 43,754 scans from two large UK hospitals for model training, and demonstrate accurate classification (area under the receiver operating curve (AUC) = 0.943) on a test set of 800 scans labelled by a team of neuroradiologists. Importantly, when trained on scans from only a single hospital the model generalized to scans from the other hospital ($Δ$AUC $\leq$ 0.02). A simulation study demonstrated that our model would reduce the mean reporting time for abnormal examinations from 28 days to 14 days and from 9 days to 5 days at the two hospitals, demonstrating feasibility for use in a clinical triage environment.
△ Less
Submitted 28 June, 2022; v1 submitted 15 June, 2021;
originally announced June 2021.
-
A Distributed Augmenting Path Approach for the Bottleneck Assignment Problem
Authors:
Mitchell Khoo,
Tony A. Wood,
Chris Manzie,
Iman Shames
Abstract:
We develop an algorithm to solve the Bottleneck Assignment Problem (BAP) that is amenable to having computation distributed over a network of agents. This consists of exploring how each component of the algorithm can be distributed, with a focus on one component in particular, i.e., the function to search for an augmenting path. An augmenting path is a common tool used in most BAP algorithms and p…
▽ More
We develop an algorithm to solve the Bottleneck Assignment Problem (BAP) that is amenable to having computation distributed over a network of agents. This consists of exploring how each component of the algorithm can be distributed, with a focus on one component in particular, i.e., the function to search for an augmenting path. An augmenting path is a common tool used in most BAP algorithms and poses a particular challenge for this distributed approach. Given this significance, we compare two different methods to search for an augmenting path in a bipartite graph. We also exploit properties of the augmenting paths to formalise conditions for which the solution from subsets of the sets of agents and tasks can be used to solve the BAP with the full sets of agents and tasks. In the end, we evaluate and compare the derived approaches with a numerical analysis.
△ Less
Submitted 18 November, 2020;
originally announced November 2020.
-
Optimization with Zeroth-Order Oracles in Formation
Authors:
Elad Michael,
Daniel Zelazo,
Tony A. Wood,
Chris Manzie,
Iman Shames
Abstract:
In this paper, we consider the optimisation of time varying functions by a network of agents with no gradient information. The proposed a novel method to estimate the gradient at each agent's position using only neighbour information. The gradient estimation is coupled with a formation controller, to minimise gradient estimation error and prevent agent collisions. Convergence results for the algor…
▽ More
In this paper, we consider the optimisation of time varying functions by a network of agents with no gradient information. The proposed a novel method to estimate the gradient at each agent's position using only neighbour information. The gradient estimation is coupled with a formation controller, to minimise gradient estimation error and prevent agent collisions. Convergence results for the algorithm are provided for functions which satisfy the Polyak-Lojasiewicz inequality. Simulations and numerical results are provided to support the theoretical results.
△ Less
Submitted 30 July, 2020;
originally announced July 2020.
-
Labelling imaging datasets on the basis of neuroradiology reports: a validation study
Authors:
David A. Wood,
Sina Kafiabadi,
Aisha Al Busaidi,
Emily Guilhem,
Jeremy Lynch,
Matthew Townend,
Antanas Montvila,
Juveria Siddiqui,
Naveen Gadapa,
Matthew Benger,
Gareth Barker,
Sebastian Ourselin,
James H. Cole,
Thomas C. Booth
Abstract:
Natural language processing (NLP) shows promise as a means to automate the labelling of hospital-scale neuroradiology magnetic resonance imaging (MRI) datasets for computer vision applications. To date, however, there has been no thorough investigation into the validity of this approach, including determining the accuracy of report labels compared to image labels as well as examining the performan…
▽ More
Natural language processing (NLP) shows promise as a means to automate the labelling of hospital-scale neuroradiology magnetic resonance imaging (MRI) datasets for computer vision applications. To date, however, there has been no thorough investigation into the validity of this approach, including determining the accuracy of report labels compared to image labels as well as examining the performance of non-specialist labellers. In this work, we draw on the experience of a team of neuroradiologists who labelled over 5000 MRI neuroradiology reports as part of a project to build a dedicated deep learning-based neuroradiology report classifier. We show that, in our experience, assigning binary labels (i.e. normal vs abnormal) to images from reports alone is highly accurate. In contrast to the binary labels, however, the accuracy of more granular labelling is dependent on the category, and we highlight reasons for this discrepancy. We also show that downstream model performance is reduced when labelling of training reports is performed by a non-specialist. To allow other researchers to accelerate their research, we make our refined abnormality definitions and labelling rules available, as well as our easy-to-use radiology report labelling app which helps streamline this process.
△ Less
Submitted 8 March, 2021; v1 submitted 8 July, 2020;
originally announced July 2020.
-
Collision Avoidance Based on Robust Lexicographic Task Assignment
Authors:
Tony A. Wood,
Mitchell Khoo,
Elad Michael,
Chris Manzie,
Iman Shames
Abstract:
Traditional task assignment approaches for multi-agent motion control do not take the possibility of collisions into account. This can lead to challenging requirements for path planning. We derive an assignment method that not only minimises the largest distance between an agent and its assigned destination but also provides local constraints for guaranteed collision avoidance. To this end, we int…
▽ More
Traditional task assignment approaches for multi-agent motion control do not take the possibility of collisions into account. This can lead to challenging requirements for path planning. We derive an assignment method that not only minimises the largest distance between an agent and its assigned destination but also provides local constraints for guaranteed collision avoidance. To this end, we introduce a sequential bottleneck optimisation problem and define a notion of robustness of an optimising assignment to changes of individual assignment costs. Conditioned on a sufficient level of robustness in relation to the size of the agents, we construct time-varying position bounds for every individual agent. These local constraints are a direct byproduct of the assignment procedure and only depend on the initial agent positions, the destinations that are to be visited, and a timing parameter. We prove that no agent that is assigned to move to one of the target locations collides with any other agent if all agents satisfy their local position constraints. We demonstrate the method in a illustrative case study.
△ Less
Submitted 17 July, 2020; v1 submitted 25 February, 2020;
originally announced February 2020.
-
Disease classification of macular Optical Coherence Tomography scans using deep learning software: validation on independent, multi-centre data
Authors:
Kanwal K. Bhatia,
Mark S. Graham,
Louise Terry,
Ashley Wood,
Paris Tranos,
Sameer Trikha,
Nicolas Jaccard
Abstract:
Purpose: To evaluate Pegasus-OCT, a clinical decision support software for the identification of features of retinal disease from macula OCT scans, across heterogenous populations involving varying patient demographics, device manufacturers, acquisition sites and operators.
Methods: 5,588 normal and anomalous macular OCT volumes (162,721 B-scans), acquired at independent centres in five countrie…
▽ More
Purpose: To evaluate Pegasus-OCT, a clinical decision support software for the identification of features of retinal disease from macula OCT scans, across heterogenous populations involving varying patient demographics, device manufacturers, acquisition sites and operators.
Methods: 5,588 normal and anomalous macular OCT volumes (162,721 B-scans), acquired at independent centres in five countries, were processed using the software. Results were evaluated against ground truth provided by the dataset owners.
Results: Pegasus-OCT performed with AUROCs of at least 98% for all datasets in the detection of general macular anomalies. For scans of sufficient quality, the AUROCs for general AMD and DME detection were found to be at least 99% and 98%, respectively.
Conclusions: The ability of a clinical decision support system to cater for different populations is key to its adoption. Pegasus-OCT was shown to be able to detect AMD, DME and general anomalies in OCT volumes acquired across multiple independent sites with high performance. Its use thus offers substantial promise, with the potential to alleviate the burden of growing demand in eye care services caused by retinal disease.
△ Less
Submitted 11 July, 2019;
originally announced July 2019.