-
Recurrent Inference Machine for Medical Image Registration
Authors:
Yi Zhang,
Yidong Zhao,
Hui Xue,
Peter Kellman,
Stefan Klein,
Qian Tao
Abstract:
Image registration is essential for medical image applications where alignment of voxels across multiple images is needed for qualitative or quantitative analysis. With recent advancements in deep neural networks and parallel computing, deep learning-based medical image registration methods become competitive with their flexible modelling and fast inference capabilities. However, compared to tradi…
▽ More
Image registration is essential for medical image applications where alignment of voxels across multiple images is needed for qualitative or quantitative analysis. With recent advancements in deep neural networks and parallel computing, deep learning-based medical image registration methods become competitive with their flexible modelling and fast inference capabilities. However, compared to traditional optimization-based registration methods, the speed advantage may come at the cost of registration performance at inference time. Besides, deep neural networks ideally demand large training datasets while optimization-based methods are training-free. To improve registration accuracy and data efficiency, we propose a novel image registration method, termed Recurrent Inference Image Registration (RIIR) network. RIIR is formulated as a meta-learning solver to the registration problem in an iterative manner. RIIR addresses the accuracy and data efficiency issues, by learning the update rule of optimization, with implicit regularization combined with explicit gradient input.
We evaluated RIIR extensively on brain MRI and quantitative cardiac MRI datasets, in terms of both registration accuracy and training data efficiency. Our experiments showed that RIIR outperformed a range of deep learning-based methods, even with only $5\%$ of the training data, demonstrating high data efficiency. Key findings from our ablation studies highlighted the important added value of the hidden states introduced in the recurrent inference framework for meta-learning. Our proposed RIIR offers a highly data-efficient framework for deep learning-based medical image registration.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Minimally Interactive Segmentation of Soft-Tissue Tumors on CT and MRI using Deep Learning
Authors:
Douwe J. Spaanderman,
Martijn P. A. Starmans,
Gonnie C. M. van Erp,
David F. Hanff,
Judith H. Sluijter,
Anne-Rose W. Schut,
Geert J. L. H. van Leenders,
Cornelis Verhoef,
Dirk J. Grunhagen,
Wiro J. Niessen,
Jacob J. Visser,
Stefan Klein
Abstract:
Segmentations are crucial in medical imaging to obtain morphological, volumetric, and radiomics biomarkers. Manual segmentation is accurate but not feasible in the radiologist's clinical workflow, while automatic segmentation generally obtains sub-par performance. We therefore developed a minimally interactive deep learning-based segmentation method for soft-tissue tumors (STTs) on CT and MRI. The…
▽ More
Segmentations are crucial in medical imaging to obtain morphological, volumetric, and radiomics biomarkers. Manual segmentation is accurate but not feasible in the radiologist's clinical workflow, while automatic segmentation generally obtains sub-par performance. We therefore developed a minimally interactive deep learning-based segmentation method for soft-tissue tumors (STTs) on CT and MRI. The method requires the user to click six points near the tumor's extreme boundaries. These six points are transformed into a distance map and serve, with the image, as input for a Convolutional Neural Network. For training and validation, a multicenter dataset containing 514 patients and nine STT types in seven anatomical locations was used, resulting in a Dice Similarity Coefficient (DSC) of 0.85$\pm$0.11 (mean $\pm$ standard deviation (SD)) for CT and 0.84$\pm$0.12 for T1-weighted MRI, when compared to manual segmentations made by expert radiologists. Next, the method was externally validated on a dataset including five unseen STT phenotypes in extremities, achieving 0.81$\pm$0.08 for CT, 0.84$\pm$0.09 for T1-weighted MRI, and 0.88\pm0.08 for previously unseen T2-weighted fat-saturated (FS) MRI. In conclusion, our minimally interactive segmentation method effectively segments different types of STTs on CT and MRI, with robust generalization to previously unseen phenotypes and imaging modalities.
△ Less
Submitted 12 February, 2024;
originally announced February 2024.
-
An Interpretable Machine Learning Model with Deep Learning-based Imaging Biomarkers for Diagnosis of Alzheimer's Disease
Authors:
Wenjie Kang,
Bo Li,
Janne M. Papma,
Lize C. Jiskoot,
Peter Paul De Deyn,
Geert Jan Biessels,
Jurgen A. H. R. Claassen,
Huub A. M. Middelkoop,
Wiesje M. van der Flier,
Inez H. G. B. Ramakers,
Stefan Klein,
Esther E. Bron
Abstract:
Machine learning methods have shown large potential for the automatic early diagnosis of Alzheimer's Disease (AD). However, some machine learning methods based on imaging data have poor interpretability because it is usually unclear how they make their decisions. Explainable Boosting Machines (EBMs) are interpretable machine learning models based on the statistical framework of generalized additiv…
▽ More
Machine learning methods have shown large potential for the automatic early diagnosis of Alzheimer's Disease (AD). However, some machine learning methods based on imaging data have poor interpretability because it is usually unclear how they make their decisions. Explainable Boosting Machines (EBMs) are interpretable machine learning models based on the statistical framework of generalized additive modeling, but have so far only been used for tabular data. Therefore, we propose a framework that combines the strength of EBM with high-dimensional imaging data using deep learning-based feature extraction. The proposed framework is interpretable because it provides the importance of each feature. We validated the proposed framework on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, achieving accuracy of 0.883 and area-under-the-curve (AUC) of 0.970 on AD and control classification. Furthermore, we validated the proposed framework on an external testing set, achieving accuracy of 0.778 and AUC of 0.887 on AD and subjective cognitive decline (SCD) classification. The proposed framework significantly outperformed an EBM model using volume biomarkers instead of deep learning-based features, as well as an end-to-end convolutional neural network (CNN) with optimized architecture.
△ Less
Submitted 15 August, 2023;
originally announced August 2023.
-
Comparison of retinal regions-of-interest imaged by OCT for the classification of intermediate AMD
Authors:
Danilo A. Jesus,
Eric F. Thee,
Tim Doekemeijer,
Daniel Luttikhuizen,
Caroline Klaver,
Stefan Klein,
Theo van Walsum,
Hans Vingerling,
Luisa Sanchez
Abstract:
To study whether it is possible to differentiate intermediate age-related macular degeneration (AMD) from healthy controls using partial optical coherence tomography (OCT) data, that is, restricting the input B-scans to certain pre-defined regions of interest (ROIs). A total of 15744 B-scans from 269 intermediate AMD patients and 115 normal subjects were used in this study (split on subject level…
▽ More
To study whether it is possible to differentiate intermediate age-related macular degeneration (AMD) from healthy controls using partial optical coherence tomography (OCT) data, that is, restricting the input B-scans to certain pre-defined regions of interest (ROIs). A total of 15744 B-scans from 269 intermediate AMD patients and 115 normal subjects were used in this study (split on subject level in 80% train, 10% validation and 10% test). From each OCT B-scan, three ROIs were extracted: retina, complex between retinal pigment epithelium (RPE) and Bruch membrane (BM), and choroid (CHO). These ROIs were obtained using two different methods: masking and crop**. In addition to the six ROIs, the whole OCT B-scan and the binary mask corresponding to the segmentation of the RPE-BM complex were used. For each subset, a convolutional neural network (based on VGG16 architecture and pre-trained on ImageNet) was trained and tested. The performance of the models was evaluated using the area under the receiver operating characteristic (AUROC), accuracy, sensitivity, and specificity. All trained models presented an AUROC, accuracy, sensitivity, and specificity equal to or higher than 0.884, 0.816, 0.685, and 0.644, respectively. The model trained on the whole OCT B-scan presented the best performance (AUROC = 0.983, accuracy = 0.927, sensitivity = 0.862, specificity = 0.913). The models trained on the ROIs obtained with the crop** method led to significantly higher outcomes than those obtained with masking, with the exception of the retinal tissue, where no statistically significant difference was observed between crop** and masking (p = 0.47). This study demonstrated that while using the complete OCT B-scan provided the highest accuracy in classifying intermediate AMD, models trained on specific ROIs such as the RPE-BM complex or the choroid can still achieve high performance.
△ Less
Submitted 14 July, 2023; v1 submitted 4 May, 2023;
originally announced May 2023.
-
Combinatorial Optimization enriched Machine Learning to solve the Dynamic Vehicle Routing Problem with Time Windows
Authors:
Léo Baty,
Kai Jungel,
Patrick S. Klein,
Axel Parmentier,
Maximilian Schiffer
Abstract:
With the rise of e-commerce and increasing customer requirements, logistics service providers face a new complexity in their daily planning, mainly due to efficiently handling same day deliveries. Existing multi-stage stochastic optimization approaches that allow to solve the underlying dynamic vehicle routing problem are either computationally too expensive for an application in online settings,…
▽ More
With the rise of e-commerce and increasing customer requirements, logistics service providers face a new complexity in their daily planning, mainly due to efficiently handling same day deliveries. Existing multi-stage stochastic optimization approaches that allow to solve the underlying dynamic vehicle routing problem are either computationally too expensive for an application in online settings, or -- in the case of reinforcement learning -- struggle to perform well on high-dimensional combinatorial problems. To mitigate these drawbacks, we propose a novel machine learning pipeline that incorporates a combinatorial optimization layer. We apply this general pipeline to a dynamic vehicle routing problem with dispatching waves, which was recently promoted in the EURO Meets NeurIPS Vehicle Routing Competition at NeurIPS 2022. Our methodology ranked first in this competition, outperforming all other approaches in solving the proposed dynamic vehicle routing problem. With this work, we provide a comprehensive numerical study that further highlights the efficacy and benefits of the proposed pipeline beyond the results achieved in the competition, e.g., by showcasing the robustness of the encoded policy against unseen instances and scenarios.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
Probabilistic 5G Indoor Positioning Proof of Concept with Outlier Rejection
Authors:
Marcus Henninger,
Traian E. Abrudan,
Silvio Mandelli,
Maximilian Arnold,
Stephan Saur,
Veli-Matti Kolmonen,
Siegfried Klein,
Thomas Schlitter,
Stephan ten Brink
Abstract:
The continuously increasing bandwidth and antenna aperture available in wireless networks laid the foundation for develo** competitive positioning solutions relying on communications standards and hardware. However, poor propagation conditions such as non-line of sight (NLOS) and rich multipath still pose many challenges due to outlier measurements that significantly degrade the positioning perf…
▽ More
The continuously increasing bandwidth and antenna aperture available in wireless networks laid the foundation for develo** competitive positioning solutions relying on communications standards and hardware. However, poor propagation conditions such as non-line of sight (NLOS) and rich multipath still pose many challenges due to outlier measurements that significantly degrade the positioning performance. In this work, we introduce an iterative positioning method that reweights the time of arrival (ToA) and angle of arrival (AoA) measurements originating from multiple locators in order to efficiently remove outliers. In contrast to existing approaches that typically rely on a single locator to set the time reference for the time difference of arrival (TDoA) measurements corresponding to the remaining locators, and whose measurements may be unreliable, the proposed iterative approach does not rely on a reference locator only. The resulting robust position estimate is then used to initialize a computationally efficient gradient search to perform maximum likelihood position estimation. Our proposal is validated with an experimental setup at 3.75 GHz with 5G numerology in an indoor factory scenario, achieving an error of less than 50 cm in 95% of the measurements. To the best of our knowledge, this paper describes the first proof of concept for 5G-based joint ToA and AoA localization.
△ Less
Submitted 18 July, 2022;
originally announced July 2022.
-
Computer-aided diagnosis and prediction in brain disorders
Authors:
Vikram Venkatraghavan,
Sebastian R. van der Voort,
Daniel Bos,
Marion Smits,
Frederik Barkhof,
Wiro J. Niessen,
Stefan Klein,
Esther E. Bron
Abstract:
Computer-aided methods have shown added value for diagnosing and predicting brain disorders and can thus support decision making in clinical care and treatment planning. This chapter will provide insight into the type of methods, their working, their input data - such as cognitive tests, imaging and genetic data - and the types of output they provide. We will focus on specific use cases for diagno…
▽ More
Computer-aided methods have shown added value for diagnosing and predicting brain disorders and can thus support decision making in clinical care and treatment planning. This chapter will provide insight into the type of methods, their working, their input data - such as cognitive tests, imaging and genetic data - and the types of output they provide. We will focus on specific use cases for diagnosis, i.e. estimating the current 'condition' of the patient, such as early detection and diagnosis of dementia, differential diagnosis of brain tumours, and decision making in stroke. Regarding prediction, i.e. estimation of the future 'condition' of the patient, we will zoom in on use cases such as predicting the disease course in multiple sclerosis and predicting patient outcomes after treatment in brain cancer. Furthermore, based on these use cases, we will assess the current state-of-the-art methodology and highlight current efforts on benchmarking of these methods and the importance of open science therein. Finally, we assess the current clinical impact of computer-aided methods and discuss the required next steps to increase clinical impact.
△ Less
Submitted 31 October, 2022; v1 submitted 29 June, 2022;
originally announced June 2022.
-
Federated Learning Enables Big Data for Rare Cancer Boundary Detection
Authors:
Sarthak Pati,
Ujjwal Baid,
Brandon Edwards,
Micah Sheller,
Shih-Han Wang,
G Anthony Reina,
Patrick Foley,
Alexey Gruzdev,
Deepthi Karkada,
Christos Davatzikos,
Chiharu Sako,
Satyam Ghodasara,
Michel Bilello,
Suyash Mohan,
Philipp Vollmuth,
Gianluca Brugnara,
Chandrakanth J Preetha,
Felix Sahm,
Klaus Maier-Hein,
Maximilian Zenk,
Martin Bendszus,
Wolfgang Wick,
Evan Calabrese,
Jeffrey Rudie,
Javier Villanueva-Meyer
, et al. (254 additional authors not shown)
Abstract:
Although machine learning (ML) has shown promise in numerous domains, there are concerns about generalizability to out-of-sample data. This is currently addressed by centrally sharing ample, and importantly diverse, data from multiple sites. However, such centralization is challenging to scale (or even not feasible) due to various limitations. Federated ML (FL) provides an alternative to train acc…
▽ More
Although machine learning (ML) has shown promise in numerous domains, there are concerns about generalizability to out-of-sample data. This is currently addressed by centrally sharing ample, and importantly diverse, data from multiple sites. However, such centralization is challenging to scale (or even not feasible) due to various limitations. Federated ML (FL) provides an alternative to train accurate and generalizable ML models, by only sharing numerical model updates. Here we present findings from the largest FL study to-date, involving data from 71 healthcare institutions across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, utilizing the largest dataset of such patients ever used in the literature (25,256 MRI scans from 6,314 patients). We demonstrate a 33% improvement over a publicly trained model to delineate the surgically targetable tumor, and 23% improvement over the tumor's entire extent. We anticipate our study to: 1) enable more studies in healthcare informed by large and diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further quantitative analyses for glioblastoma via performance optimization of our consensus model for eventual public release, and 3) demonstrate the effectiveness of FL at such scale and task complexity as a paradigm shift for multi-site collaborations, alleviating the need for data sharing.
△ Less
Submitted 25 April, 2022; v1 submitted 22 April, 2022;
originally announced April 2022.
-
Multi-Atlas Segmentation and Spatial Alignment of the Human Embryo in First Trimester 3D Ultrasound
Authors:
W. A. P. Bastiaansen,
M. Rousian,
R. P. M. Steegers-Theunissen,
W. J. Niessen,
A. H. J. Koning,
S. Klein
Abstract:
Segmentation and spatial alignment of ultrasound (US) imaging data acquired in the in first trimester are crucial for monitoring human embryonic growth and development throughout this crucial period of life. Current approaches are either manual or semi-automatic and are therefore very time-consuming and prone to errors. To automate these tasks, we propose a multi-atlas framework for automatic segm…
▽ More
Segmentation and spatial alignment of ultrasound (US) imaging data acquired in the in first trimester are crucial for monitoring human embryonic growth and development throughout this crucial period of life. Current approaches are either manual or semi-automatic and are therefore very time-consuming and prone to errors. To automate these tasks, we propose a multi-atlas framework for automatic segmentation and spatial alignment of the embryo using deep learning with minimal supervision. Our framework learns to register the embryo to an atlas, which consists of the US images acquired at a range of gestational age (GA), segmented and spatially aligned to a predefined standard orientation. From this, we can derive the segmentation of the embryo and put the embryo in standard orientation. US images acquired at 8+0 till 12+6 weeks GA were used and eight subjects were selected as atlas. We evaluated different fusion strategies to incorporate multiple atlases: 1) training the framework using atlas images from a single subject, 2) training the framework with data of all available atlases and 3) ensembling of the frameworks trained per subject. To evaluate the performance, we calculated the Dice score over the test set. We found that training the framework using all available atlases outperformed ensembling and gave similar results compared to the best of all frameworks trained on a single subject. Furthermore, we found that selecting images from the four atlases closest in GA out of all available atlases, regardless of the individual quality, gave the best results with a median Dice score of 0.72. We conclude that our framework can accurately segment and spatially align the embryo in first trimester 3D US images and is robust for the variation in quality that existed in the available atlases.
△ Less
Submitted 28 August, 2023; v1 submitted 14 February, 2022;
originally announced February 2022.
-
Electric vehicle charge scheduling with flexible service operations
Authors:
Patrick Sean Klein,
Maximilian Schiffer
Abstract:
Operators who deploy large fleets of electric vehicles often face a challenging charge scheduling problem. Specifically, time-ineffective recharging operations limit the profitability of charging during service operations such that operators recharge vehicles off-duty at a central depot. Here, high investment cost and grid capacity limit available charging infrastructure such that operators need t…
▽ More
Operators who deploy large fleets of electric vehicles often face a challenging charge scheduling problem. Specifically, time-ineffective recharging operations limit the profitability of charging during service operations such that operators recharge vehicles off-duty at a central depot. Here, high investment cost and grid capacity limit available charging infrastructure such that operators need to schedule charging operations to keep the fleet operational. In this context, flexible service operations, i.e. allowing to delay or expedite vehicle departures, can potentially increase charger utilization. Beyond this, jointly scheduling charging and service operations promises operational cost savings through better utilization of Time-of-Use energy tariffs and carefully crafted charging schedules designed to minimize battery wear. Against this background, we study the resulting joint charging and service operations scheduling problem accounting for battery degradation, non-linear charging, and Time-of-Use energy tariffs. We propose an exact Branch & Price algorithm, leveraging a custom branching rule and a primal heuristic to remain efficient during the Branch & Bound phase. Moreover, we develop an exact labeling algorithm for our pricing problem, constituting a resource-constrained shortest path problem that considers variable energy prices and non-linear charging operations. We benchmark our algorithm in a comprehensive numerical study and show that it can solve problem instances of realistic size with computational times below one hour, thus enabling its application in practice. Additionally, we analyze the benefit of jointly scheduling charging and service operations. We find that our integrated approach lowers the amount of charging infrastructure required by up to 57% besides enabling operational cost savings of up to 5%.
△ Less
Submitted 18 August, 2022; v1 submitted 11 January, 2022;
originally announced January 2022.
-
Automatic Segmentation of the Optic Nerve Head Region in Optical Coherence Tomography: A Methodological Review
Authors:
Rita Marques,
Danilo Andrade De Jesus,
João Barbosa Breda,
Jan Van Eijgen,
Ingeborg Stalmans,
Theo van Walsum,
Stefan Klein,
Pedro G. Vaz,
Luisa Sánchez Brea
Abstract:
The optic nerve head represents the intraocular section of the optic nerve (ONH), which is prone to damage by intraocular pressure. The advent of optical coherence tomography (OCT) has enabled the evaluation of novel optic nerve head parameters, namely the depth and curvature of the lamina cribrosa (LC). Together with the Bruch's membrane opening minimum-rim-width, these seem to be promising optic…
▽ More
The optic nerve head represents the intraocular section of the optic nerve (ONH), which is prone to damage by intraocular pressure. The advent of optical coherence tomography (OCT) has enabled the evaluation of novel optic nerve head parameters, namely the depth and curvature of the lamina cribrosa (LC). Together with the Bruch's membrane opening minimum-rim-width, these seem to be promising optic nerve head parameters for diagnosis and monitoring of retinal diseases such as glaucoma. Nonetheless, these optical coherence tomography derived biomarkers are mostly extracted through manual segmentation, which is time-consuming and prone to bias, thus limiting their usability in clinical practice. The automatic segmentation of optic nerve head in OCT scans could further improve the current clinical management of glaucoma and other diseases.
This review summarizes the current state-of-the-art in automatic segmentation of the ONH in OCT. PubMed and Scopus were used to perform a systematic review. Additional works from other databases (IEEE, Google Scholar and ARVO IOVS) were also included, resulting in a total of 27 reviewed studies.
For each algorithm, the methods, the size and type of dataset used for validation, and the respective results were carefully analyzed. The results show that deep learning-based algorithms provide the highest accuracy, sensitivity and specificity for segmenting the different structures of the ONH including the LC. However, a lack of consensus regarding the definition of segmented regions, extracted parameters and validation approaches has been observed, highlighting the importance and need of standardized methodologies for ONH segmentation.
△ Less
Submitted 6 September, 2021;
originally announced September 2021.
-
Signal-carrying speckle in Optical Coherence Tomography: a methodological review on biomedical applications
Authors:
Vania Bastos Silva,
Danilo Andrade De Jesus,
Stefan Klein,
Theo van Walsum,
João Cardoso,
Luisa Sánchez Brea,
Pedro G. Vaz
Abstract:
Significance: Speckle has historically been considered a source of noise in coherent light imaging. However, a number of works in optical coherence tomography (OCT) imaging have shown that speckle patterns may contain relevant information regarding sub-resolution and structural properties of the tissues from which it is originated.
Aim: The objective of this work is to provide a comprehensive ov…
▽ More
Significance: Speckle has historically been considered a source of noise in coherent light imaging. However, a number of works in optical coherence tomography (OCT) imaging have shown that speckle patterns may contain relevant information regarding sub-resolution and structural properties of the tissues from which it is originated.
Aim: The objective of this work is to provide a comprehensive overview of the methods developed for retrieving speckle information in biomedical OCT applications.
Approach: PubMed and Scopus databases were used to perform a systematic review on studies published until April 2021. From 134-screened studies, 37 were eligible for this review.
Results: The studies have been clustered according to the nature of their analysis, namely static or dynamic, and all features were described and analysed. The results show that features retrieved from speckle can be used successfully in different applications, such as classification and segmentation. However, the results also show that speckle analysis is highly application-dependant, and the best approach varies between applications.
Conclusions: Several of the reviewed analysis were only performed in a theoretical context or using phantoms, showing that signal-carrying speckle analysis in OCT imaging is still in its early stage, and further work is needed to validate its applicability and reproducibility in a clinical context.
△ Less
Submitted 30 August, 2021;
originally announced August 2021.
-
Reproducible radiomics through automated machine learning validated on twelve clinical applications
Authors:
Martijn P. A. Starmans,
Sebastian R. van der Voort,
Thomas Phil,
Milea J. M. Timbergen,
Melissa Vos,
Guillaume A. Padmos,
Wouter Kessels,
David Hanff,
Dirk J. Grunhagen,
Cornelis Verhoef,
Stefan Sleijfer,
Martin J. van den Bent,
Marion Smits,
Roy S. Dwarkasing,
Christopher J. Els,
Federico Fiduzi,
Geert J. L. H. van Leenders,
Anela Blazevic,
Johannes Hofland,
Tessa Brabander,
Renza A. H. van Gils,
Gaston J. H. Franssen,
Richard A. Feelders,
Wouter W. de Herder,
Florian E. Buisman
, et al. (21 additional authors not shown)
Abstract:
Radiomics uses quantitative medical imaging features to predict clinical outcomes. Currently, in a new clinical application, finding the optimal radiomics method out of the wide range of available options has to be done manually through a heuristic trial-and-error process. In this study we propose a framework for automatically optimizing the construction of radiomics workflows per application. To…
▽ More
Radiomics uses quantitative medical imaging features to predict clinical outcomes. Currently, in a new clinical application, finding the optimal radiomics method out of the wide range of available options has to be done manually through a heuristic trial-and-error process. In this study we propose a framework for automatically optimizing the construction of radiomics workflows per application. To this end, we formulate radiomics as a modular workflow and include a large collection of common algorithms for each component. To optimize the workflow per application, we employ automated machine learning using a random search and ensembling. We evaluate our method in twelve different clinical applications, resulting in the following area under the curves: 1) liposarcoma (0.83); 2) desmoid-type fibromatosis (0.82); 3) primary liver tumors (0.80); 4) gastrointestinal stromal tumors (0.77); 5) colorectal liver metastases (0.61); 6) melanoma metastases (0.45); 7) hepatocellular carcinoma (0.75); 8) mesenteric fibrosis (0.80); 9) prostate cancer (0.72); 10) glioma (0.71); 11) Alzheimer's disease (0.87); and 12) head and neck cancer (0.84). We show that our framework has a competitive performance compared human experts, outperforms a radiomics baseline, and performs similar or superior to Bayesian optimization and more advanced ensemble approaches. Concluding, our method fully automatically optimizes the construction of radiomics workflows, thereby streamlining the search for radiomics biomarkers in new applications. To facilitate reproducibility and future research, we publicly release six datasets, the software implementation of our framework, and the code to reproduce this study.
△ Less
Submitted 29 July, 2022; v1 submitted 19 August, 2021;
originally announced August 2021.
-
Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
Authors:
Richard Osuala,
Kaisar Kushibar,
Lidia Garrucho,
Akis Linardos,
Zuzanna Szafranowska,
Stefan Klein,
Ben Glocker,
Oliver Diaz,
Karim Lekadir
Abstract:
Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in Generative Adversarial Networks…
▽ More
Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in Generative Adversarial Networks (GANs), data synthesis, and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.
△ Less
Submitted 27 November, 2022; v1 submitted 20 July, 2021;
originally announced July 2021.
-
Recurrent Inference Machines as inverse problem solvers for MR relaxometry
Authors:
E. R. Sabidussi,
S. Klein,
M. W. A. Caan,
S. Bazrafkan,
A. J. den Dekker,
J. Sijbers,
W. J. Niessen,
D. H. J. Poot
Abstract:
In this paper, we propose the use of Recurrent Inference Machines (RIMs) to perform T1 and T2 map**. The RIM is a neural network framework that learns an iterative inference process based on the signal model, similar to conventional statistical methods for quantitative MRI (QMRI), such as the Maximum Likelihood Estimator (MLE). This framework combines the advantages of both data-driven and model…
▽ More
In this paper, we propose the use of Recurrent Inference Machines (RIMs) to perform T1 and T2 map**. The RIM is a neural network framework that learns an iterative inference process based on the signal model, similar to conventional statistical methods for quantitative MRI (QMRI), such as the Maximum Likelihood Estimator (MLE). This framework combines the advantages of both data-driven and model-based methods, and, we hypothesize, is a promising tool for QMRI. Previously, RIMs were used to solve linear inverse reconstruction problems. Here, we show that they can also be used to optimize non-linear problems and estimate relaxometry maps with high precision and accuracy. The developed RIM framework is evaluated in terms of accuracy and precision and compared to an MLE method and an implementation of the ResNet. The results show that the RIM improves the quality of estimates compared to the other techniques in Monte Carlo experiments with simulated data, test-retest analysis of a system phantom, and in-vivo scans. Additionally, inference with the RIM is 150 times faster than the MLE, and robustness to (slight) variations of scanning parameters is demonstrated. Hence, the RIM is a promising and flexible method for QMRI. Coupled with an open-source training data generation tool, it presents a compelling alternative to previous methods.
△ Less
Submitted 8 June, 2021;
originally announced June 2021.
-
Evaluating glioma growth predictions as a forward ranking problem
Authors:
Karin A. van Garderen,
Sebastian R. van der Voort,
Maarten M. J. Wijnenga,
Fatih Incekara,
Georgios Kapsas,
Renske Gahrmann,
Ahmad Alafandi,
Marion Smits,
Stefan Klein
Abstract:
The problem of tumor growth prediction is challenging, but promising results have been achieved with both model-driven and statistical methods. In this work, we present a framework for the evaluation of growth predictions that focuses on the spatial infiltration patterns, and specifically evaluating a prediction of future growth. We propose to frame the problem as a ranking problem rather than a s…
▽ More
The problem of tumor growth prediction is challenging, but promising results have been achieved with both model-driven and statistical methods. In this work, we present a framework for the evaluation of growth predictions that focuses on the spatial infiltration patterns, and specifically evaluating a prediction of future growth. We propose to frame the problem as a ranking problem rather than a segmentation problem. Using the average precision as a metric, we can evaluate the results with segmentations while using the full spatiotemporal prediction. Furthermore, by separating the model goodness-of-fit from future predictive performance, we show that in some cases, a better fit of model parameters does not guarantee a better the predictive power.
△ Less
Submitted 22 March, 2021;
originally announced March 2021.
-
Longitudinal diffusion MRI analysis using Segis-Net: a single-step deep-learning framework for simultaneous segmentation and registration
Authors:
Bo Li,
Wiro J. Niessen,
Stefan Klein,
Marius de Groot,
M. Arfan Ikram,
Meike W. Vernooij,
Esther E. Bron
Abstract:
This work presents a single-step deep-learning framework for longitudinal image analysis, coined Segis-Net. To optimally exploit information available in longitudinal data, this method concurrently learns a multi-class segmentation and nonlinear registration. Segmentation and registration are modeled using a convolutional neural network and optimized simultaneously for their mutual benefit. An obj…
▽ More
This work presents a single-step deep-learning framework for longitudinal image analysis, coined Segis-Net. To optimally exploit information available in longitudinal data, this method concurrently learns a multi-class segmentation and nonlinear registration. Segmentation and registration are modeled using a convolutional neural network and optimized simultaneously for their mutual benefit. An objective function that optimizes spatial correspondence for the segmented structures across time-points is proposed. We applied Segis-Net to the analysis of white matter tracts from N=8045 longitudinal brain MRI datasets of 3249 elderly individuals. Segis-Net approach showed a significant increase in registration accuracy, spatio-temporal segmentation consistency, and reproducibility comparing with two multistage pipelines. This also led to a significant reduction in the sample-size that would be required to achieve the same statistical power in analyzing tract-specific measures. Thus, we expect that Segis-Net can serve as a new reliable tool to support longitudinal imaging studies to investigate macro- and microstructural brain changes over time.
△ Less
Submitted 23 April, 2021; v1 submitted 28 December, 2020;
originally announced December 2020.
-
Cross-Cohort Generalizability of Deep and Conventional Machine Learning for MRI-based Diagnosis and Prediction of Alzheimer's Disease
Authors:
Esther E. Bron,
Stefan Klein,
Janne M. Papma,
Lize C. Jiskoot,
Vikram Venkatraghavan,
Jara Linders,
Pauline Aalten,
Peter Paul De Deyn,
Geert Jan Biessels,
Jurgen A. H. R. Claassen,
Huub A. M. Middelkoop,
Marion Smits,
Wiro J. Niessen,
John C. van Swieten,
Wiesje M. van der Flier,
Inez H. G. B. Ramakers,
Aad van der Lugt
Abstract:
This work validates the generalizability of MRI-based classification of Alzheimer's disease (AD) patients and controls (CN) to an external data set and to the task of prediction of conversion to AD in individuals with mild cognitive impairment (MCI). We used a conventional support vector machine (SVM) and a deep convolutional neural network (CNN) approach based on structural MRI scans that underwe…
▽ More
This work validates the generalizability of MRI-based classification of Alzheimer's disease (AD) patients and controls (CN) to an external data set and to the task of prediction of conversion to AD in individuals with mild cognitive impairment (MCI). We used a conventional support vector machine (SVM) and a deep convolutional neural network (CNN) approach based on structural MRI scans that underwent either minimal pre-processing or more extensive pre-processing into modulated gray matter (GM) maps. Classifiers were optimized and evaluated using cross-validation in the ADNI (334 AD, 520 CN). Trained classifiers were subsequently applied to predict conversion to AD in ADNI MCI patients (231 converters, 628 non-converters) and in the independent Health-RI Parelsnoer data set. From this multi-center study representing a tertiary memory clinic population, we included 199 AD patients, 139 participants with subjective cognitive decline, 48 MCI patients converting to dementia, and 91 MCI patients who did not convert to dementia. AD-CN classification based on modulated GM maps resulted in a similar AUC for SVM (0.940) and CNN (0.933). Application to conversion prediction in MCI yielded significantly higher performance for SVM (0.756) than for CNN (0.742). In external validation, performance was slightly decreased. For AD-CN, it again gave similar AUCs for SVM (0.896) and CNN (0.876). For prediction in MCI, performances decreased for both SVM (0.665) and CNN (0.702). Both with SVM and CNN, classification based on modulated GM maps significantly outperformed classification based on minimally processed images. Deep and conventional classifiers performed equally well for AD classification and their performance decreased only slightly when applied to the external cohort. We expect that this work on external validation contributes towards translation of machine learning to clinical practice.
△ Less
Submitted 26 May, 2021; v1 submitted 16 December, 2020;
originally announced December 2020.
-
Learning unbiased group-wise registration (LUGR) and joint segmentation: evaluation on longitudinal diffusion MRI
Authors:
Bo Li,
Wiro J. Niessen,
Stefan Klein,
M. Arfan Ikram,
Meike W. Vernooij,
Esther E. Bron
Abstract:
Analysis of longitudinal changes in imaging studies often involves both segmentation of structures of interest and registration of multiple timeframes. The accuracy of such analysis could benefit from a tailored framework that jointly optimizes both tasks to fully exploit the information available in the longitudinal data. Most learning-based registration algorithms, including joint optimization a…
▽ More
Analysis of longitudinal changes in imaging studies often involves both segmentation of structures of interest and registration of multiple timeframes. The accuracy of such analysis could benefit from a tailored framework that jointly optimizes both tasks to fully exploit the information available in the longitudinal data. Most learning-based registration algorithms, including joint optimization approaches, currently suffer from bias due to selection of a fixed reference frame and only support pairwise transformations. We here propose an analytical framework based on an unbiased learning strategy for group-wise registration that simultaneously registers images to the mean space of a group to obtain consistent segmentations. We evaluate the proposed method on longitudinal analysis of a white matter tract in a brain MRI dataset with 2-3 time-points for 3249 individuals, i.e., 8045 images in total. The reproducibility of the method is evaluated on test-retest data from 97 individuals. The results confirm that the implicit reference image is an average of the input image. In addition, the proposed framework leads to consistent segmentations and significantly lower processing bias than that of a pair-wise fixed-reference approach. This processing bias is even smaller than those obtained when translating segmentations by only one voxel, which can be attributed to subtle numerical instabilities and interpolation. Therefore, we postulate that the proposed mean-space learning strategy could be widely applied to learning-based registration tasks. In addition, this group-wise framework introduces a novel way for learning-based longitudinal studies by direct construction of an unbiased within-subject template and allowing reliable and efficient analysis of spatio-temporal imaging biomarkers.
△ Less
Submitted 24 February, 2021; v1 submitted 3 November, 2020;
originally announced November 2020.
-
Differential diagnosis and molecular stratification of gastrointestinal stromal tumors on CT images using a radiomics approach
Authors:
Martijn P. A. Starmans,
Milea J. M. Timbergen,
Melissa Vos,
Michel Renckens,
Dirk J. Grünhagen,
Geert J. L. H. van Leenders,
Roy S. Dwarkasing,
François E. J. A. Willemssen,
Wiro J. Niessen,
Cornelis Verhoef,
Stefan Sleijfer,
Jacob J. Visser,
Stefan Klein
Abstract:
Distinguishing gastrointestinal stromal tumors (GISTs) from other intra-abdominal tumors and GISTs molecular analysis is necessary for treatment planning, but challenging due to its rarity. The aim of this study was to evaluate radiomics for distinguishing GISTs from other intra-abdominal tumors, and in GISTs, predict the c-KIT, PDGFRA,BRAF mutational status and mitotic index (MI). All 247 include…
▽ More
Distinguishing gastrointestinal stromal tumors (GISTs) from other intra-abdominal tumors and GISTs molecular analysis is necessary for treatment planning, but challenging due to its rarity. The aim of this study was to evaluate radiomics for distinguishing GISTs from other intra-abdominal tumors, and in GISTs, predict the c-KIT, PDGFRA,BRAF mutational status and mitotic index (MI). All 247 included patients (125 GISTS, 122 non-GISTs) underwent a contrast-enhanced venous phase CT. The GIST vs. non-GIST radiomics model, including imaging, age, sex and location, had a mean area under the curve (AUC) of 0.82. Three radiologists had an AUC of 0.69, 0.76, and 0.84, respectively. The radiomics model had an AUC of 0.52 for c-KIT, 0.56 for c-KIT exon 11, and 0.52 for the MI. Hence, our radiomics model was able to distinguish GIST from non-GISTS with a performance similar to three radiologists, but was not able to predict the c-KIT mutation or MI.
△ Less
Submitted 15 October, 2020; v1 submitted 14 October, 2020;
originally announced October 2020.
-
Time efficiency analysis for undersampled quantitative MRI acquisitions
Authors:
Riwaj Byanju,
Stefan Klein,
Alexandra Cristobal-Huerta,
J. A. Hernandez-Tamames,
Dirk H. J. Poot
Abstract:
To realize Quantitative MRI (QMRI) with clinically acceptable scan time, acceleration factors achieved by conventional parallel imaging techniques are often inadequate. Further acceleration is possible using model-based reconstruction. We propose a theoretical metric called TEUSQA: Time Efficiency for UnderSampled QMRI Acquisitions to inform sequence design and sample pattern optimisation. TEUSQA…
▽ More
To realize Quantitative MRI (QMRI) with clinically acceptable scan time, acceleration factors achieved by conventional parallel imaging techniques are often inadequate. Further acceleration is possible using model-based reconstruction. We propose a theoretical metric called TEUSQA: Time Efficiency for UnderSampled QMRI Acquisitions to inform sequence design and sample pattern optimisation. TEUSQA is designed for a particular class of reconstruction techniques that directly estimate tissue parameters, possibly using prior information to regularize the estimation. TEUSQA can be used to evaluate undersampling patterns for multi-contrast QMRI sequences targeting any tissue parameter. To verify the time efficiency predicted by TEUSQA, we performed Monte Carlo simulations and an accelerated parameter map** with two sequences (Inversion prepared fast spin echo for T1 and T2 map** and 3D GRASE for T2 and B0 inhomogeneity map**). Using TEUSQA, we assessed several ways to generate undersampling patterns in silico, providing insight into the relation between sample distribution and time efficiency for different acceleration factors. The time efficiency predicted by TEUSQA was within 15 % of that observed in the Monte Carlo simulations and the prospective acquisition experiment. The assessment of undersampling patterns showed that a class of good patterns could be obtained by low-discrepancy sampling. We believe that TEUSQA offers a valuable instrument for developers of novel QMRI sequences pushing the boundaries of acceleration to achieve clinically feasible protocols. Finally, we applied a time-efficient undersampling pattern selected using TEUSQA for a 32-fold accelerated scan to map T1 and T2 map** of a healthy volunteer.
△ Less
Submitted 22 December, 2021; v1 submitted 13 October, 2020;
originally announced October 2020.
-
WHO 2016 subty** and automated segmentation of glioma using multi-task deep learning
Authors:
Sebastian R. van der Voort,
Fatih Incekara,
Maarten M. J. Wijnenga,
Georgios Kapsas,
Renske Gahrmann,
Joost W. Schouten,
Rishi Nandoe Tewarie,
Geert J. Lycklama,
Philip C. De Witt Hamer,
Roelant S. Eijgelaar,
Pim J. French,
Hendrikus J. Dubbink,
Arnaud J. P. E. Vincent,
Wiro J. Niessen,
Martin J. van den Bent,
Marion Smits,
Stefan Klein
Abstract:
Accurate characterization of glioma is crucial for clinical decision making. A delineation of the tumor is also desirable in the initial decision stages but is a time-consuming task. Leveraging the latest GPU capabilities, we developed a single multi-task convolutional neural network that uses the full 3D, structural, pre-operative MRI scans to can predict the IDH mutation status, the 1p/19q co-de…
▽ More
Accurate characterization of glioma is crucial for clinical decision making. A delineation of the tumor is also desirable in the initial decision stages but is a time-consuming task. Leveraging the latest GPU capabilities, we developed a single multi-task convolutional neural network that uses the full 3D, structural, pre-operative MRI scans to can predict the IDH mutation status, the 1p/19q co-deletion status, and the grade of a tumor, while simultaneously segmenting the tumor. We trained our method using the largest, most diverse patient cohort to date containing 1508 glioma patients from 16 institutes. We tested our method on an independent dataset of 240 patients from 13 different institutes, and achieved an IDH-AUC of 0.90, 1p/19q-AUC of 0.85, grade-AUC of 0.81, and a mean whole tumor DICE score of 0.84. Thus, our method non-invasively predicts multiple, clinically relevant parameters and generalizes well to the broader clinical population.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
-
Towards segmentation and spatial alignment of the human embryonic brain using deep learning for atlas-based registration
Authors:
Wietske A. P. Bastiaansen,
Melek Rousian,
Régine P. M. Steegers-Theunissen,
Wiro J. Niessen,
Anton Koning,
Stefan Klein
Abstract:
We propose an unsupervised deep learning method for atlas based registration to achieve segmentation and spatial alignment of the embryonic brain in a single framework. Our approach consists of two sequential networks with a specifically designed loss function to address the challenges in 3D first trimester ultrasound. The first part learns the affine transformation and the second part learns the…
▽ More
We propose an unsupervised deep learning method for atlas based registration to achieve segmentation and spatial alignment of the embryonic brain in a single framework. Our approach consists of two sequential networks with a specifically designed loss function to address the challenges in 3D first trimester ultrasound. The first part learns the affine transformation and the second part learns the voxelwise nonrigid deformation between the target image and the atlas. We trained this network end-to-end and validated it against a ground truth on synthetic datasets designed to resemble the challenges present in 3D first trimester ultrasound. The method was tested on a dataset of human embryonic ultrasound volumes acquired at 9 weeks gestational age, which showed alignment of the brain in some cases and gave insight in open challenges for the proposed method. We conclude that our method is a promising approach towards fully automated spatial alignment and segmentation of embryonic brains in 3D ultrasound.
△ Less
Submitted 13 May, 2020;
originally announced May 2020.
-
TADPOLE Challenge: Accurate Alzheimer's disease prediction through crowdsourced forecasting of future data
Authors:
Razvan V. Marinescu,
Neil P. Oxtoby,
Alexandra L. Young,
Esther E. Bron,
Arthur W. Toga,
Michael W. Weiner,
Frederik Barkhof,
Nick C. Fox,
Polina Golland,
Stefan Klein,
Daniel C. Alexander
Abstract:
The TADPOLE Challenge compares the performance of algorithms at predicting the future evolution of individuals at risk of Alzheimer's disease. TADPOLE Challenge participants train their models and algorithms on historical data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. Participants are then required to make forecasts of three key outcomes for ADNI-3 rollover participants: c…
▽ More
The TADPOLE Challenge compares the performance of algorithms at predicting the future evolution of individuals at risk of Alzheimer's disease. TADPOLE Challenge participants train their models and algorithms on historical data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) study. Participants are then required to make forecasts of three key outcomes for ADNI-3 rollover participants: clinical diagnosis, ADAS-Cog 13, and total volume of the ventricles -- which are then compared with future measurements. Strong points of the challenge are that the test data did not exist at the time of forecasting (it was acquired afterwards), and that it focuses on the challenging problem of cohort selection for clinical trials by identifying fast progressors. The submission phase of TADPOLE was open until 15 November 2017; since then data has been acquired until April 2019 from 219 subjects with 223 clinical visits and 150 Magnetic Resonance Imaging (MRI) scans, which was used for the evaluation of the participants' predictions. Thirty-three teams participated with a total of 92 submissions. No single submission was best at predicting all three outcomes. For diagnosis prediction, the best forecast (team Frog), which was based on gradient boosting, obtained a multiclass area under the receiver-operating curve (MAUC) of 0.931, while for ventricle prediction the best forecast (team EMC1), which was based on disease progression modelling and spline regression, obtained mean absolute error of 0.41% of total intracranial volume (ICV). For ADAS-Cog 13, no forecast was considerably better than the benchmark mixed effects model (BenchmarkME), provided to participants before the submission deadline. Further analysis can help understand which input features and algorithms are most suitable for Alzheimer's disease prediction and for aiding patient stratification in clinical trials.
△ Less
Submitted 23 January, 2020;
originally announced January 2020.
-
An Efficient Method for Multi-Parameter Map** in Quantitative MRI using B-Spline Interpolation
Authors:
Willem van Valenberg,
Stefan Klein,
Frans M. Vos,
Kirsten Koolstra,
Lucas J. van Vliet,
Dirk H. J. Poot
Abstract:
Quantitative MRI methods that estimate multiple physical parameters simultaneously often require the fitting of a computational complex signal model defined through the Bloch equations. Repeated Bloch simulations can be avoided by matching the measured signal with a precomputed signal dictionary on a discrete parameter grid, as used in MR Fingerprinting. However, accurate estimation requires discr…
▽ More
Quantitative MRI methods that estimate multiple physical parameters simultaneously often require the fitting of a computational complex signal model defined through the Bloch equations. Repeated Bloch simulations can be avoided by matching the measured signal with a precomputed signal dictionary on a discrete parameter grid, as used in MR Fingerprinting. However, accurate estimation requires discretizing each parameter with a high resolution and consequently high computational and memory costs for dictionary generation, storage, and matching. Here, we reduce the required parameter resolution by approximating the signal between grid points through B-spline interpolation. The interpolant and its gradient are evaluated efficiently which enables a least-squares fitting method for parameter map**. The resolution of each parameter was minimized while obtaining a user-specified interpolation accuracy. The method was evaluated by phantom and in-vivo experiments using fully-sampled and undersampled unbalanced (FISP) MR fingerprinting acquisitions. Bloch simulations incorporated relaxation effects ($T_1,T_2$), proton density ($PD$), receiver phase ($φ_0$), transmit field inhomogeneity ($B_1^+$), and slice profile. Parameter maps were compared with those obtained from dictionary matching, where the parameter resolution was chosen to obtain similar signal (interpolation) accuracy. For both the phantom and the in-vivo acquisition, the proposed method approximated the parameter maps obtained through dictionary matching while reducing the parameter resolution in each dimension ($T_1,T_2,B_1^+$) by, on average, an order of magnitude. In effect, the applied dictionary was reduced from 1.47 GB to 464 KB. Dictionary fitting with B-spline interpolation reduces the computational and memory costs of dictionary-based methods and is therefore a promising method for multi-parametric map**.
△ Less
Submitted 18 November, 2019;
originally announced November 2019.
-
Towards continuous learning for glioma segmentation with elastic weight consolidation
Authors:
Karin van Garderen,
Sebastian van der Voort,
Fatih Incekara,
Marion Smits,
Stefan Klein
Abstract:
When finetuning a convolutional neural network (CNN) on data from a new domain, catastrophic forgetting will reduce performance on the original training data. Elastic Weight Consolidation (EWC) is a recent technique to prevent this, which we evaluated while training and re-training a CNN to segment glioma on two different datasets. The network was trained on the public BraTS dataset and finetuned…
▽ More
When finetuning a convolutional neural network (CNN) on data from a new domain, catastrophic forgetting will reduce performance on the original training data. Elastic Weight Consolidation (EWC) is a recent technique to prevent this, which we evaluated while training and re-training a CNN to segment glioma on two different datasets. The network was trained on the public BraTS dataset and finetuned on an in-house dataset with non-enhancing low-grade glioma. EWC was found to decrease catastrophic forgetting in this case, but was also found to restrict adaptation to the new domain.
△ Less
Submitted 25 September, 2019;
originally announced September 2019.
-
APIR-Net: Autocalibrated Parallel Imaging Reconstruction using a Neural Network
Authors:
Chao** Zhang,
Florian Dubost,
Marleen de Bruijne,
Stefan Klein,
Dirk H. J. Poot
Abstract:
Deep learning has been successfully demonstrated in MRI reconstruction of accelerated acquisitions. However, its dependence on representative training data limits the application across different contrasts, anatomies, or image sizes. To address this limitation, we propose an unsupervised, auto-calibrated k-space completion method, based on a uniquely designed neural network that reconstructs the f…
▽ More
Deep learning has been successfully demonstrated in MRI reconstruction of accelerated acquisitions. However, its dependence on representative training data limits the application across different contrasts, anatomies, or image sizes. To address this limitation, we propose an unsupervised, auto-calibrated k-space completion method, based on a uniquely designed neural network that reconstructs the full k-space from an undersampled k-space, exploiting the redundancy among the multiple channels in the receive coil in a parallel imaging acquisition. To achieve this, contrary to common convolutional network approaches, the proposed network has a decreasing number of feature maps of constant size. In contrast to conventional parallel imaging methods such as GRAPPA that estimate the prediction kernel from the fully sampled autocalibration signals in a linear way, our method is able to learn nonlinear relations between sampled and unsampled positions in k-space. The proposed method was compared to the start-of-the-art ESPIRiT and RAKI methods in terms of noise amplification and visual image quality in both phantom and in-vivo experiments. The experiments indicate that APIR-Net provides a promising alternative to the conventional parallel imaging methods, and results in improved image quality especially for low SNR acquisitions.
△ Less
Submitted 19 September, 2019;
originally announced September 2019.
-
A hybrid deep learning framework for integrated segmentation and registration: evaluation on longitudinal white matter tract changes
Authors:
Bo Li,
Wiro Niessen,
Stefan Klein,
Marius de Groot,
Arfan Ikram,
Meike Vernooij,
Esther Bron
Abstract:
To accurately analyze changes of anatomical structures in longitudinal imaging studies, consistent segmentation across multiple time-points is required. Existing solutions often involve independent registration and segmentation components. Registration between time-points is used either as a prior for segmentation in a subsequent time point or to perform segmentation in a common space. In this wor…
▽ More
To accurately analyze changes of anatomical structures in longitudinal imaging studies, consistent segmentation across multiple time-points is required. Existing solutions often involve independent registration and segmentation components. Registration between time-points is used either as a prior for segmentation in a subsequent time point or to perform segmentation in a common space. In this work, we propose a novel hybrid convolutional neural network (CNN) that integrates segmentation and registration into a single procedure. We hypothesize that the joint optimization leads to increased performance on both tasks. The hybrid CNN is trained by minimizing an integrated loss function composed of four different terms, measuring segmentation accuracy, similarity between registered images, deformation field smoothness, and segmentation consistency. We applied this method to the segmentation of white matter tracts, describing functionally grouped axonal fibers, using N=8045 longitudinal brain MRI data of 3249 individuals. The proposed method was compared with two multistage pipelines using two existing segmentation methods combined with a conventional deformable registration algorithm. In addition, we assessed the added value of the joint optimization for segmentation and registration separately. The hybrid CNN yielded significantly higher accuracy, consistency and reproducibility of segmentation than the multistage pipelines, and was orders of magnitude faster. Therefore, we expect it can serve as a novel tool to support clinical and epidemiological analyses on understanding microstructural brain changes over time.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation
Authors:
Egor Panfilov,
Aleksei Tiulpin,
Stefan Klein,
Miika T. Nieminen,
Simo Saarakkala
Abstract:
Degeneration of articular cartilage (AC) is actively studied in knee osteoarthritis (OA) research via magnetic resonance imaging (MRI). Segmentation of AC tissues from MRI data is an essential step in quantification of their damage. Deep learning (DL) based methods have shown potential in this realm and are the current state-of-the-art, however, their robustness to heterogeneity of MRI acquisition…
▽ More
Degeneration of articular cartilage (AC) is actively studied in knee osteoarthritis (OA) research via magnetic resonance imaging (MRI). Segmentation of AC tissues from MRI data is an essential step in quantification of their damage. Deep learning (DL) based methods have shown potential in this realm and are the current state-of-the-art, however, their robustness to heterogeneity of MRI acquisition settings remains an open problem. In this study, we investigated two modern regularization techniques -- mixup and adversarial unsupervised domain adaptation (UDA) -- to improve the robustness of DL-based knee cartilage segmentation to new MRI acquisition settings. Our validation setup included two datasets produced by different MRI scanners and using distinct data acquisition protocols. We assessed the robustness of automatic segmentation by comparing mixup and UDA approaches to a strong baseline method at different OA severity stages and, additionally, in relation to anatomical locations. Our results showed that for moderate changes in knee MRI data acquisition settings both approaches may provide notable improvements in the robustness, which are consistent for all stages of the disease and affect the clinically important areas of the knee joint. However, mixup may be considered as a recommended approach, since it is more computationally efficient and does not require additional data from the target acquisition setup.
△ Less
Submitted 27 October, 2019; v1 submitted 12 August, 2019;
originally announced August 2019.
