-
MONAI: An open-source framework for deep learning in healthcare
Authors:
M. Jorge Cardoso,
Wenqi Li,
Richard Brown,
Nic Ma,
Eric Kerfoot,
Yiheng Wang,
Benjamin Murrey,
Andriy Myronenko,
Can Zhao,
Dong Yang,
Vishwesh Nath,
Yufan He,
Ziyue Xu,
Ali Hatamizadeh,
Andriy Myronenko,
Wentao Zhu,
Yun Liu,
Mingxin Zheng,
Yucheng Tang,
Isaac Yang,
Michael Zephyr,
Behrooz Hashemian,
Sachidanand Alle,
Mohammad Zalbagi Darestani,
Charlie Budd
, et al. (32 additional authors not shown)
Abstract:
Artificial Intelligence (AI) is having a tremendous impact across most areas of science. Applications of AI in healthcare have the potential to improve our ability to detect, diagnose, prognose, and intervene on human disease. For AI models to be used clinically, they need to be made safe, reproducible and robust, and the underlying software framework must be aware of the particularities (e.g. geo…
▽ More
Artificial Intelligence (AI) is having a tremendous impact across most areas of science. Applications of AI in healthcare have the potential to improve our ability to detect, diagnose, prognose, and intervene on human disease. For AI models to be used clinically, they need to be made safe, reproducible and robust, and the underlying software framework must be aware of the particularities (e.g. geometry, physiology, physics) of medical data being processed. This work introduces MONAI, a freely available, community-supported, and consortium-led PyTorch-based framework for deep learning in healthcare. MONAI extends PyTorch to support medical data, with a particular focus on imaging, and provide purpose-specific AI model architectures, transformations and utilities that streamline the development and deployment of medical AI models. MONAI follows best practices for software-development, providing an easy-to-use, robust, well-documented, and well-tested software framework. MONAI preserves the simple, additive, and compositional approach of its underlying PyTorch libraries. MONAI is being used by and receiving contributions from research, clinical and industrial teams from around the world, who are pursuing applications spanning nearly every aspect of healthcare.
△ Less
Submitted 4 November, 2022;
originally announced November 2022.
-
Optimal Transport Features for Morphometric Population Analysis
Authors:
Samuel Gerber,
Marc Niethammer,
Ebrahim Ebrahim,
Joseph Piven,
Stephen R. Dager,
Martin Styner,
Stephen Aylward,
Andinet Enquobahrie
Abstract:
Brain pathologies often manifest as partial or complete loss of tissue. The goal of many neuroimaging studies is to capture the location and amount of tissue changes with respect to a clinical variable of interest, such as disease progression. Morphometric analysis approaches capture local differences in the distribution of tissue or other quantities of interest in relation to a clinical variable.…
▽ More
Brain pathologies often manifest as partial or complete loss of tissue. The goal of many neuroimaging studies is to capture the location and amount of tissue changes with respect to a clinical variable of interest, such as disease progression. Morphometric analysis approaches capture local differences in the distribution of tissue or other quantities of interest in relation to a clinical variable. We propose to augment morphometric analysis with an additional feature extraction step based on unbalanced optimal transport. The optimal transport feature extraction step increases statistical power for pathologies that cause spatially dispersed tissue loss, minimizes sensitivity to shifts due to spatial misalignment or differences in brain topology, and separates changes due to volume differences from changes due to tissue location. We demonstrate the proposed optimal transport feature extraction step in the context of a volumetric morphometric analysis of the OASIS-1 study for Alzheimer's disease. The results demonstrate that the proposed approach can identify tissue changes and differences that are not otherwise measurable.
△ Less
Submitted 11 August, 2022;
originally announced August 2022.
-
Fluid registration between lung CT and stationary chest tomosynthesis images
Authors:
Lin Tian,
Connor Puett,
Peirong Liu,
Zhengyang Shen,
Stephen R. Aylward,
Yueh Z. Lee,
Marc Niethammer
Abstract:
Registration is widely used in image-guided therapy and image-guided surgery to estimate spatial correspondences between organs of interest between planning and treatment images. However, while high-quality computed tomography (CT) images are often available at planning time, limited angle acquisitions are frequently used during treatment because of radiation concerns or imaging time constraints.…
▽ More
Registration is widely used in image-guided therapy and image-guided surgery to estimate spatial correspondences between organs of interest between planning and treatment images. However, while high-quality computed tomography (CT) images are often available at planning time, limited angle acquisitions are frequently used during treatment because of radiation concerns or imaging time constraints. This requires algorithms to register CT images based on limited angle acquisitions. We, therefore, formulate a 3D/2D registration approach which infers a 3D deformation based on measured projections and digitally reconstructed radiographs of the CT. Most 3D/2D registration approaches use simple transformation models or require complex mathematical derivations to formulate the underlying optimization problem. Instead, our approach entirely relies on differentiable operations which can be combined with modern computational toolboxes supporting automatic differentiation. This then allows for rapid prototy**, integration with deep neural networks, and to support a variety of transformation models including fluid flow models. We demonstrate our approach for the registration between CT and stationary chest tomosynthesis (sDCT) images and show how it naturally leads to an iterative image reconstruction approach.
△ Less
Submitted 6 March, 2022;
originally announced March 2022.
-
Deep Decomposition for Stochastic Normal-Abnormal Transport
Authors:
Peirong Liu,
Yueh Lee,
Stephen Aylward,
Marc Niethammer
Abstract:
Advection-diffusion equations describe a large family of natural transport processes, e.g., fluid flow, heat transfer, and wind transport. They are also used for optical flow and perfusion imaging computations. We develop a machine learning model, D^2-SONATA, built upon a stochastic advection-diffusion equation, which predicts the velocity and diffusion fields that drive 2D/3D image time-series of…
▽ More
Advection-diffusion equations describe a large family of natural transport processes, e.g., fluid flow, heat transfer, and wind transport. They are also used for optical flow and perfusion imaging computations. We develop a machine learning model, D^2-SONATA, built upon a stochastic advection-diffusion equation, which predicts the velocity and diffusion fields that drive 2D/3D image time-series of transport. In particular, our proposed model incorporates a model of transport atypicality, which isolates abnormal differences between expected normal transport behavior and the observed transport. In a medical context such a normal-abnormal decomposition can be used, for example, to quantify pathologies. Specifically, our model identifies the advection and diffusion contributions from the transport time-series and simultaneously predicts an anomaly value field to provide a decomposition into normal and abnormal advection and diffusion behavior. To achieve improved estimation performance for the velocity and diffusion-tensor fields underlying the advection-diffusion process and for the estimation of the anomaly fields, we create a 2D/3D anomaly-encoded advection-diffusion simulator, which allows for supervised learning. We further apply our model on a brain perfusion dataset from ischemic stroke patients via transfer learning. Extensive comparisons demonstrate that our model successfully distinguishes stroke lesions (abnormal) from normal brain regions, while reconstructing the underlying velocity and diffusion tensor fields.
△ Less
Submitted 28 March, 2022; v1 submitted 29 November, 2021;
originally announced November 2021.
-
Discovering Hidden Physics Behind Transport Dynamics
Authors:
Peirong Liu,
Lin Tian,
Yubo Zhang,
Stephen R. Aylward,
Yueh Z. Lee,
Marc Niethammer
Abstract:
Transport processes are ubiquitous. They are, for example, at the heart of optical flow approaches; or of perfusion imaging, where blood transport is assessed, most commonly by injecting a tracer. An advection-diffusion equation is widely used to describe these transport phenomena. Our goal is estimating the underlying physics of advection-diffusion equations, expressed as velocity and diffusion t…
▽ More
Transport processes are ubiquitous. They are, for example, at the heart of optical flow approaches; or of perfusion imaging, where blood transport is assessed, most commonly by injecting a tracer. An advection-diffusion equation is widely used to describe these transport phenomena. Our goal is estimating the underlying physics of advection-diffusion equations, expressed as velocity and diffusion tensor fields. We propose a learning framework (YETI) building on an auto-encoder structure between 2D and 3D image time-series, which incorporates the advection-diffusion model. To help with identifiability, we develop an advection-diffusion simulator which allows pre-training of our model by supervised learning using the velocity and diffusion tensor fields. Instead of directly learning these velocity and diffusion tensor fields, we introduce representations that assure incompressible flow and symmetric positive semi-definite diffusion fields and demonstrate the additional benefits of these representations on improving estimation accuracy. We further use transfer learning to apply YETI on a public brain magnetic resonance (MR) perfusion dataset of stroke patients and show its ability to successfully distinguish stroke lesions from normal brain regions via the estimated velocity and diffusion tensor fields.
△ Less
Submitted 29 March, 2021; v1 submitted 24 November, 2020;
originally announced November 2020.
-
Perfusion Imaging: A Data Assimilation Approach
Authors:
Peirong Liu,
Yueh Z. Lee,
Stephen R. Aylward,
Marc Niethammer
Abstract:
Perfusion imaging (PI) is clinically used to assess strokes and brain tumors. Commonly used PI approaches based on magnetic resonance imaging (MRI) or computed tomography (CT) measure the effect of a contrast agent moving through blood vessels and into tissue. Contrast-agent free approaches, for example, based on intravoxel incoherent motion, also exist, but are so far not routinely used clinicall…
▽ More
Perfusion imaging (PI) is clinically used to assess strokes and brain tumors. Commonly used PI approaches based on magnetic resonance imaging (MRI) or computed tomography (CT) measure the effect of a contrast agent moving through blood vessels and into tissue. Contrast-agent free approaches, for example, based on intravoxel incoherent motion, also exist, but are so far not routinely used clinically. These methods rely on estimating on the arterial input function (AIF) to approximately model tissue perfusion, neglecting spatial dependencies, and reliably estimating the AIF is also non-trivial, leading to difficulties with standardizing perfusion measures. In this work we therefore propose a data-assimilation approach (PIANO) which estimates the velocity and diffusion fields of an advection-diffusion model that best explains the contrast dynamics. PIANO accounts for spatial dependencies and neither requires estimating the AIF nor relies on a particular contrast agent bolus shape. Specifically, we propose a convenient parameterization of the estimation problem, a numerical estimation approach, and extensively evaluate PIANO. We demonstrate that PIANO can successfully resolve velocity and diffusion field ambiguities and results in sensitive measures for the assessment of stroke, comparing favorably to conventional measures of perfusion.
△ Less
Submitted 6 September, 2020;
originally announced September 2020.
-
A Framework for Challenge Design: Insight and Deployment Challenges to Address Medical Image Analysis Problems
Authors:
Adriƫnne M. Mendrik,
Stephen R. Aylward
Abstract:
In this paper we aim to refine the concept of grand challenges in medical image analysis, based on statistical principles from quantitative and qualitative experimental research. We identify two types of challenges based on their generalization objective: 1) a deployment challenge and 2) an insight challenge. A deployment challenge's generalization objective is to find algorithms that solve a medi…
▽ More
In this paper we aim to refine the concept of grand challenges in medical image analysis, based on statistical principles from quantitative and qualitative experimental research. We identify two types of challenges based on their generalization objective: 1) a deployment challenge and 2) an insight challenge. A deployment challenge's generalization objective is to find algorithms that solve a medical image analysis problem, which thereby requires the use of a quantitative experimental design. An insight challenge's generalization objective is to gain a broad understanding of what class of algorithms might be effective for a class of medical image analysis problems, in which case a qualitative experimental design is sufficient. Both challenge types are valuable, but problems arise when a challenge's design and objective are inconsistent, as is often the case when a challenge does not carefully consider these concepts. Therefore, in this paper, we propose a theoretical framework, based on statistical principles, to guide researchers in challenge design, documentation, and assessment. Experimental results are given that explore the factors that effect the practical implementation of this theoretical framework.
△ Less
Submitted 19 November, 2019;
originally announced November 2019.
-
Beyond the Leaderboard: Insight and Deployment Challenges to Address Research Problems
Authors:
Adrienne M. Mendrik,
Stephen R. Aylward
Abstract:
In the medical image analysis field, organizing challenges with associated workshops at international conferences began in 2007 and has grown to include over 150 challenges. Several of these challenges have had a major impact in the field. However, whereas well-designed challenges have the potential to unite and focus the field on creating solutions to important problems, poorly designed and docum…
▽ More
In the medical image analysis field, organizing challenges with associated workshops at international conferences began in 2007 and has grown to include over 150 challenges. Several of these challenges have had a major impact in the field. However, whereas well-designed challenges have the potential to unite and focus the field on creating solutions to important problems, poorly designed and documented challenges can equally impede a field and lead to pursuing incremental improvements in metric scores with no theoretic or clinical significance. This is supported by a critical assessment of challenges at the international MICCAI conference. In this assessment the main observation was that small changes to the underlying challenge data can drastically change the ranking order on the leaderboard. Related to this is the practice of leaderboard climbing, which is characterized by participants focusing on incrementally improving metric results rather than advancing science or solving the driving problem of a challenge. In this abstract we look beyond the leaderboard of a challenge and instead look at the conclusions that can be drawn from a challenge with respect to the research problem that it is addressing. Research study design is well described in other research areas and can be translated to challenge design when viewing challenges as research studies on algorithm performance that address a research problem. Based on the two main types of scientific research study design, we propose two main challenge types, which we think would benefit other research areas as well: 1) an insight challenge that is based on a qualitative study design and 2) a deployment challenge that is based on a quantitative study design. In addition we briefly touch upon related considerations with respect to statistical significance versus practical significance, generalizability and data saturation.
△ Less
Submitted 26 October, 2018;
originally announced November 2018.
-
Brain Extraction from Normal and Pathological Images: A Joint PCA/Image-Reconstruction Approach
Authors:
Xu Han,
Roland Kwitt,
Stephen Aylward,
Spyridon Bakas,
Bjoern Menze,
Alexander Asturias,
Paul Vespa,
John Van Horn,
Marc Niethammer
Abstract:
Brain extraction from images is a common pre-processing step. Many approaches exist, but they are frequently only designed to perform brain extraction from images without strong pathologies. Extracting the brain from images with strong pathologies, for example, the presence of a tumor or of a traumatic brain injury, is challenging. In such cases, tissue appearance may deviate from normal tissue an…
▽ More
Brain extraction from images is a common pre-processing step. Many approaches exist, but they are frequently only designed to perform brain extraction from images without strong pathologies. Extracting the brain from images with strong pathologies, for example, the presence of a tumor or of a traumatic brain injury, is challenging. In such cases, tissue appearance may deviate from normal tissue and violates algorithmic assumptions for these approaches; hence, the brain may not be correctly extracted. This paper proposes a brain extraction approach which can explicitly account for pathologies by jointly modeling normal tissue and pathologies. Specifically, our model uses a three-part image decomposition: (1) normal tissue appearance is captured by principal component analysis, (2) pathologies are captured via a total variation term, and (3) non-brain tissue is captured by a sparse term. Decomposition and image registration steps are alternated to allow statistical modeling in a fixed atlas space. As a beneficial side effect, the model allows for the identification of potential pathologies and the reconstruction of a quasi-normal image in atlas space. We demonstrate the effectiveness of our method on four datasets: the IBSR and LPBA40 datasets which show normal images, the BRATS dataset containing images with brain tumors and a dataset containing clinical TBI images. We compare the performance with other popular models: ROBEX, BEaST, MASS, BET, BSE and a recently proposed deep learning approach. Our model performs better than these competing methods on all four datasets. Specifically, our model achieves the best median (97.11) and mean (96.88) Dice scores over all datasets. The two best performing competitors, ROBEX and MASS, achieve scores of 96.23/95.62 and 96.67/94.25 respectively. Hence, our approach is an effective method for high quality brain extraction on a wide variety of images.
△ Less
Submitted 30 April, 2018; v1 submitted 15 November, 2017;
originally announced November 2017.
-
Efficient Registration of Pathological Images: A Joint PCA/Image-Reconstruction Approach
Authors:
Xu Han,
Xiao Yang,
Stephen Aylward,
Roland Kwitt,
Marc Niethammer
Abstract:
Registration involving one or more images containing pathologies is challenging, as standard image similarity measures and spatial transforms cannot account for common changes due to pathologies. Low-rank/Sparse (LRS) decomposition removes pathologies prior to registration; however, LRS is memory-demanding and slow, which limits its use on larger data sets. Additionally, LRS blurs normal tissue re…
▽ More
Registration involving one or more images containing pathologies is challenging, as standard image similarity measures and spatial transforms cannot account for common changes due to pathologies. Low-rank/Sparse (LRS) decomposition removes pathologies prior to registration; however, LRS is memory-demanding and slow, which limits its use on larger data sets. Additionally, LRS blurs normal tissue regions, which may degrade registration performance. This paper proposes an efficient alternative to LRS: (1) normal tissue appearance is captured by principal component analysis (PCA) and (2) blurring is avoided by an integrated model for pathology removal and image reconstruction. Results on synthetic and BRATS 2015 data demonstrate its utility.
△ Less
Submitted 31 March, 2017;
originally announced April 2017.