-
Deep Learning-based Intraoperative MRI Reconstruction
Authors:
Jon André Ottesen,
Tryggve Storas,
Svein Are Sirirud Vatnehol,
Grethe Løvland,
Einar O. Vik-Mo,
Till Schellhorn,
Karoline Skogen,
Christopher Larsson,
Atle Bjørnerud,
Inge Rasmus Groote-Eindbaas,
Matthan W. A. Caan
Abstract:
Purpose: To evaluate the quality of deep learning reconstruction for prospectively accelerated intraoperative magnetic resonance imaging (iMRI) during resective brain tumor surgery.
Materials and Methods: Accelerated iMRI was performed during brain surgery using dual surface coils positioned around the area of resection. A deep learning (DL) model was trained on the fastMRI neuro dataset to mimi…
▽ More
Purpose: To evaluate the quality of deep learning reconstruction for prospectively accelerated intraoperative magnetic resonance imaging (iMRI) during resective brain tumor surgery.
Materials and Methods: Accelerated iMRI was performed during brain surgery using dual surface coils positioned around the area of resection. A deep learning (DL) model was trained on the fastMRI neuro dataset to mimic the data from the iMRI protocol. Evaluation was performed on imaging material from 40 patients imaged between 01.11.2021 - 01.06.2023 that underwent iMRI during tumor resection surgery. A comparative analysis was conducted between the conventional compressed sense (CS) method and the trained DL reconstruction method. Blinded evaluation of multiple image quality metrics was performed by two working neuro-radiologists and a working neurosurgeon on a 1 to 5 Likert scale (1=non diagnostic, 2=poor, 3=acceptable, 4=good, 5=excellent), and the favored reconstruction variant.
Results: The DL reconstruction was strongly favored or favored over the CS reconstruction for 33/40, 39/40, and 8/40 of cases for reader 1, 2, and 3, respectively. Two of three readers consistently assigned higher ratings for the DL reconstructions, and the DL reconstructions had a higher score than their respective CS counterparts for 72%, 72%, and 14% of the cases for reader 1, 2, and 3, respectively. Still, the DL reconstructions exhibited shortcomings such as a stri** artifact and reduced signal.
Conclusion: DL shows promise to allow for high-quality reconstructions of intraoperative MRI with equal to or improved perceived spatial resolution, signal-to-noise ratio, diagnostic confidence, diagnostic conspicuity, and spatial resolution compared to compressed sense.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Treatment-aware Diffusion Probabilistic Model for Longitudinal MRI Generation and Diffuse Glioma Growth Prediction
Authors:
Qinghui Liu,
Elies Fuster-Garcia,
Ivar Thokle Hovden,
Donatas Sederevicius,
Karoline Skogen,
Bradley J MacIntosh,
Edvard Grødem,
Till Schellhorn,
Petter Brandal,
Atle Bjørnerud,
Kyrre Eeg Emblem
Abstract:
Diffuse gliomas are malignant brain tumors that grow widespread through the brain. The complex interactions between neoplastic cells and normal tissue, as well as the treatment-induced changes often encountered, make glioma tumor growth modeling challenging. In this paper, we present a novel end-to-end network capable of generating future tumor masks and realistic MRIs of how the tumor will look a…
▽ More
Diffuse gliomas are malignant brain tumors that grow widespread through the brain. The complex interactions between neoplastic cells and normal tissue, as well as the treatment-induced changes often encountered, make glioma tumor growth modeling challenging. In this paper, we present a novel end-to-end network capable of generating future tumor masks and realistic MRIs of how the tumor will look at any future time points for different treatment plans. Our approach is based on cutting-edge diffusion probabilistic models and deep-segmentation neural networks. We included sequential multi-parametric magnetic resonance images (MRI) and treatment information as conditioning inputs to guide the generative diffusion process. This allows for tumor growth estimates at any given time point. We trained the model using real-world postoperative longitudinal MRI data with glioma tumor growth trajectories represented as tumor segmentation maps over time. The model has demonstrated promising performance across a range of tasks, including the generation of high-quality synthetic MRIs with tumor masks, time-series tumor segmentations, and uncertainty estimates. Combined with the treatment-aware generated MRIs, the tumor growth predictions with uncertainty estimates can provide useful information for clinical decision-making.
△ Less
Submitted 14 September, 2023; v1 submitted 11 September, 2023;
originally announced September 2023.
-
Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets
Authors:
Sheng He,
Rina Bao,
**gpeng Li,
Jeffrey Stout,
Atle Bjornerud,
P. Ellen Grant,
Yangming Ou
Abstract:
Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images. It comes without previously-required re-training or fine-tuning specific to each new dataset.
Purpose: To test SAM's accuracy in various medical image segmentation tasks and investigate potential factors that may affect its accurac…
▽ More
Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images. It comes without previously-required re-training or fine-tuning specific to each new dataset.
Purpose: To test SAM's accuracy in various medical image segmentation tasks and investigate potential factors that may affect its accuracy in medical images.
Methods: SAM was tested on 12 public medical image segmentation datasets involving 7,451 subjects. The accuracy was measured by the Dice overlap between the algorithm-segmented and ground-truth masks. SAM was compared with five state-of-the-art algorithms specifically designed for medical image segmentation tasks. Associations of SAM's accuracy with six factors were computed, independently and jointly, including segmentation difficulties as measured by segmentation ability score and by Dice overlap in U-Net, image dimension, size of the target region, image modality, and contrast.
Results: The Dice overlaps from SAM were significantly lower than the five medical-image-based algorithms in all 12 medical image segmentation datasets, by a margin of 0.1-0.5 and even 0.6-0.7 Dice. SAM-Semantic was significantly associated with medical image segmentation difficulty and the image modality, and SAM-Point and SAM-Box were significantly associated with image segmentation difficulty, image dimension, target region size, and target-vs-background contrast. All these 3 variations of SAM were more accurate in 2D medical images, larger target region sizes, easier cases with a higher Segmentation Ability score and higher U-Net Dice, and higher foreground-background contrast.
△ Less
Submitted 5 May, 2023; v1 submitted 18 April, 2023;
originally announced April 2023.
-
Voxels Intersecting along Orthogonal Levels Attention U-Net for Intracerebral Haemorrhage Segmentation in Head CT
Authors:
Qinghui Liu,
Bradley J MacIntosh,
Till Schellhorn,
Karoline Skogen,
KyrreEeg Emblem,
Atle Bjørnerud
Abstract:
We propose a novel and flexible attention based U-Net architecture referred to as "Voxels-Intersecting Along Orthogonal Levels Attention U-Net" (viola-Unet), for intracranial hemorrhage (ICH) segmentation task in the INSTANCE 2022 Data Challenge on non-contrast computed tomography (CT). The performance of ICH segmentation was improved by efficiently incorporating fused spatially orthogonal and cro…
▽ More
We propose a novel and flexible attention based U-Net architecture referred to as "Voxels-Intersecting Along Orthogonal Levels Attention U-Net" (viola-Unet), for intracranial hemorrhage (ICH) segmentation task in the INSTANCE 2022 Data Challenge on non-contrast computed tomography (CT). The performance of ICH segmentation was improved by efficiently incorporating fused spatially orthogonal and cross-channel features via our proposed Viola attention plugged into the U-Net decoding branches. The viola-Unet outperformed the strong baseline nnU-Net models during both 5-fold cross validation and online validation. Our solution was the winner of the challenge validation phase in terms of all four performance metrics (i.e., DSC, HD, NSD, and RVD). The code base, pretrained weights, and docker image of the viola-Unet AI tool are publicly available at \url{https://github.com/samleoqh/Viola-Unet}.
△ Less
Submitted 4 April, 2023; v1 submitted 12 August, 2022;
originally announced August 2022.
-
Segmenting white matter hyperintensities on isotropic three-dimensional Fluid Attenuated Inversion Recovery magnetic resonance images: Assessing deep learning tools on norwegian imaging database
Authors:
Martin Soria Røvang,
Per Selnes,
Bradley John MacIntosh,
Inge Rasmus Groote,
Lene Paalhaugen,
Sudre Carole,
Tormod Fladby,
Atle Bjørnerud
Abstract:
Automated segmentation of white matter hyperintensities (WMHs) is an essential step in neuroimaging analysis of Magnetic Resonance Imaging (MRI). Fluid Attenuated Inversion Recovery (FLAIR-weighted) is an MRI contrast that is particularly useful to visualize and quantify WMHs, a hallmark of cerebral small vessel disease and Alzheimer's disease (AD). Clinical MRI protocols migrate to a three-dimens…
▽ More
Automated segmentation of white matter hyperintensities (WMHs) is an essential step in neuroimaging analysis of Magnetic Resonance Imaging (MRI). Fluid Attenuated Inversion Recovery (FLAIR-weighted) is an MRI contrast that is particularly useful to visualize and quantify WMHs, a hallmark of cerebral small vessel disease and Alzheimer's disease (AD). Clinical MRI protocols migrate to a three-dimensional (3D) FLAIR-weighted acquisition to enable high spatial resolution in all three voxel dimensions. The current study details the deployment of deep learning tools to enable automated WMH segmentation and characterization from 3D FLAIR-weighted images acquired as part of a national AD imaging initiative.
Among 441 participants (194 male, mean age: (64.91 +/- 9.32) years) from the DDI study, two in-house networks were trained and validated across five national collection sites. Three models were tested on a held-out subset of the internal data from the 441 participants and an external dataset with 29 cases from an international collaborator. These test sets were evaluated independently. Five established WMH performance metrics were used for comparison against ground truth human-in-the-loop segmentation.
Results of the three networks tested, the 3D nnU-Net had the best performance with an average dice similarity coefficient score of 0.76 +/- 0.16, performing better than both the in-house developed 2.5D model and the SOTA Deep Bayesian network.
With the increasing use of 3D FLAIR-weighted images in MRI protocols, our results suggest that WMH segmentation models can be trained on 3D data and yield WMH segmentation performance that is comparable to or better than state-of-the-art without the need for including T1-weighted image series.
△ Less
Submitted 11 May, 2023; v1 submitted 18 July, 2022;
originally announced July 2022.
-
A Densely Interconnected Network for Deep Learning Accelerated MRI
Authors:
Jon Andre Ottesen,
Matthan W. A. Caan,
Inge Rasmus Groote,
Atle Bjørnerud
Abstract:
Objective: To improve accelerated MRI reconstruction through a densely connected cascading deep learning reconstruction framework.
Materials and Methods: A cascading deep learning reconstruction framework (baseline model) was modified by applying three architectural modifications: Input-level dense connections between cascade inputs and outputs, an improved deep learning sub-network, and long-ra…
▽ More
Objective: To improve accelerated MRI reconstruction through a densely connected cascading deep learning reconstruction framework.
Materials and Methods: A cascading deep learning reconstruction framework (baseline model) was modified by applying three architectural modifications: Input-level dense connections between cascade inputs and outputs, an improved deep learning sub-network, and long-range skip-connections between subsequent deep learning networks. An ablation study was performed, where five model configurations were trained on the NYU fastMRI neuro dataset with an end-to-end scheme conjunct on four- and eight-fold acceleration. The trained models were evaluated by comparing their respective structural similarity index measure (SSIM), normalized mean square error (NMSE) and peak signal to noise ratio (PSNR).
Results: The proposed densely interconnected residual cascading network (DIRCN), utilizing all three suggested modifications, achieved a SSIM improvement of 8% and 11% for four- and eight-fold acceleration, respectively. For eight-fold acceleration, the model achieved a 23% decrease in the NMSE when compared to the baseline model. In an ablation study, the individual architectural modifications all contributed to this improvement, by reducing the SSIM and NMSE with approximately 3% and 5% for four-fold acceleration, respectively.
Conclusion: The proposed architectural modifications allow for simple adjustments on an already existing cascading framework to further improve the resulting reconstructions.
△ Less
Submitted 22 May, 2023; v1 submitted 5 July, 2022;
originally announced July 2022.
