Showing 1–25 of 25 results for author: Freiman, M

Leveraging Prompt-Learning for Structured Information Extraction from Crohn's Disease Radiology Reports in a Low-Resource Language

Authors: Liam Hazan, Gili Focht, Naama Gavrielov, Roi Reichart, Talar Hagopian, Mary-Louise C. Greer, Ruth Cytter Kuint, Dan Turner, Moti Freiman

Abstract: Automatic conversion of free-text radiology reports into structured data using Natural Language Processing (NLP) techniques is crucial for analyzing diseases on a large scale. While effective for tasks in widely spoken languages like English, generative large language models (LLMs) typically underperform with less common languages and can pose potential risks to patient privacy. Fine-tuning local… ▽ More Automatic conversion of free-text radiology reports into structured data using Natural Language Processing (NLP) techniques is crucial for analyzing diseases on a large scale. While effective for tasks in widely spoken languages like English, generative large language models (LLMs) typically underperform with less common languages and can pose potential risks to patient privacy. Fine-tuning local NLP models is hindered by the skewed nature of real-world medical datasets, where rare findings represent a significant data imbalance. We introduce SMP-BERT, a novel prompt learning method that leverages the structured nature of reports to overcome these challenges. In our studies involving a substantial collection of Crohn's disease radiology reports in Hebrew (over 8,000 patients and 10,000 reports), SMP-BERT greatly surpassed traditional fine-tuning methods in performance, notably in detecting infrequent conditions (AUC: 0.99 vs 0.94, F1: 0.84 vs 0.34). SMP-BERT empowers more accurate AI diagnostics available for low-resource languages. △ Less

Submitted 22 May, 2024; v1 submitted 2 May, 2024; originally announced May 2024.

Automated Prediction of Breast Cancer Response to Neoadjuvant Chemotherapy from DWI Data

Authors: Shir Nitzan, Maya Gilad, Moti Freiman

Abstract: Effective surgical planning for breast cancer hinges on accurately predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC). Diffusion-weighted MRI (DWI) and machine learning offer a non-invasive approach for early pCR assessment. However, most machine-learning models require manual tumor segmentation, a cumbersome and error-prone task. We propose a deep learning model emp… ▽ More Effective surgical planning for breast cancer hinges on accurately predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC). Diffusion-weighted MRI (DWI) and machine learning offer a non-invasive approach for early pCR assessment. However, most machine-learning models require manual tumor segmentation, a cumbersome and error-prone task. We propose a deep learning model employing "Size-Adaptive Lesion Weighting" for automatic DWI tumor segmentation to enhance pCR prediction accuracy. Despite histopathological changes during NAC complicating DWI image segmentation, our model demonstrates robust performance. Utilizing the BMMR2 challenge dataset, it matches human experts in pCR prediction pre-NAC with an area under the curve (AUC) of 0.76 vs. 0.796, and surpasses standard automated methods mid-NAC, with an AUC of 0.729 vs. 0.654 and 0.576. Our approach represents a significant advancement in automating breast cancer treatment planning, enabling more reliable pCR predictions without manual segmentation. △ Less

Submitted 7 April, 2024; originally announced April 2024.

Comments: Accepted for presentation at the IEEE International Symposium on Biomedical Imaging (ISBI)

NPB-REC: A Non-parametric Bayesian Deep-learning Approach for Undersampled MRI Reconstruction with Uncertainty Estimation

Authors: Samah Khawaled, Moti Freiman

Abstract: The ability to reconstruct high-quality images from undersampled MRI data is vital in improving MRI temporal resolution and reducing acquisition times. Deep learning methods have been proposed for this task, but the lack of verified methods to quantify the uncertainty in the reconstructed images hampered clinical applicability. We introduce "NPB-REC", a non-parametric fully Bayesian framework, for… ▽ More The ability to reconstruct high-quality images from undersampled MRI data is vital in improving MRI temporal resolution and reducing acquisition times. Deep learning methods have been proposed for this task, but the lack of verified methods to quantify the uncertainty in the reconstructed images hampered clinical applicability. We introduce "NPB-REC", a non-parametric fully Bayesian framework, for MRI reconstruction from undersampled data with uncertainty estimation. We use Stochastic Gradient Langevin Dynamics during training to characterize the posterior distribution of the network parameters. This enables us to both improve the quality of the reconstructed images and quantify the uncertainty in the reconstructed images. We demonstrate the efficacy of our approach on a multi-coil MRI dataset from the fastMRI challenge and compare it to the baseline End-to-End Variational Network (E2E-VarNet). Our approach outperforms the baseline in terms of reconstruction accuracy by means of PSNR and SSIM ($34.55$, $0.908$ vs. $33.08$, $0.897$, $p<0.01$, acceleration rate $R=8$) and provides uncertainty measures that correlate better with the reconstruction error (Pearson correlation, $R=0.94$ vs. $R=0.91$). Additionally, our approach exhibits better generalization capabilities against anatomical distribution shifts (PSNR and SSIM of $32.38$, $0.849$ vs. $31.63$, $0.836$, $p<0.01$, training on brain data, inference on knee data, acceleration rate $R=8$). NPB-REC has the potential to facilitate the safe utilization of deep learning-based methods for MRI reconstruction from undersampled data. Code and trained models are available at \url{https://github.com/samahkh/NPB-REC}. △ Less

Submitted 6 April, 2024; originally announced April 2024.

Comments: Published in Artificial Intelligence in Medicine, DOI: https://doi.org/10.1016/j.artmed.2024.102798 This is an extension representing a more comprehensive work extending preliminary work presented at arXiv:2208.03966

A self-attention model for robust rigid slice-to-volume registration of functional MRI

Authors: Samah Khawaled, Simon K. Warfield, Moti Freiman

Abstract: Functional Magnetic Resonance Imaging (fMRI) is vital in neuroscience, enabling investigations into brain disorders, treatment monitoring, and brain function map**. However, head motion during fMRI scans, occurring between shots of slice acquisition, can result in distortion, biased analyses, and increased costs due to the need for scan repetitions. Therefore, retrospective slice-level motion co… ▽ More Functional Magnetic Resonance Imaging (fMRI) is vital in neuroscience, enabling investigations into brain disorders, treatment monitoring, and brain function map**. However, head motion during fMRI scans, occurring between shots of slice acquisition, can result in distortion, biased analyses, and increased costs due to the need for scan repetitions. Therefore, retrospective slice-level motion correction through slice-to-volume registration (SVR) is crucial. Previous studies have utilized deep learning (DL) based models to address the SVR task; however, they overlooked the uncertainty stemming from the input stack of slices and did not assign weighting or scoring to each slice. In this work, we introduce an end-to-end SVR model for aligning 2D fMRI slices with a 3D reference volume, incorporating a self-attention mechanism to enhance robustness against input data variations and uncertainties. It utilizes independent slice and volume encoders and a self-attention module to assign pixel-wise scores for each slice. We conducted evaluation experiments on 200 images involving synthetic rigid motion generated from 27 subjects belonging to the test set, from the publicly available Healthy Brain Network (HBN) dataset. Our experimental results demonstrate that our model achieves competitive performance in terms of alignment accuracy compared to state-of-the-art deep learning-based methods (Euclidean distance of $0.93$ [mm] vs. $1.86$ [mm]). Furthermore, our approach exhibits significantly faster registration speed compared to conventional iterative methods ($0.096$ sec. vs. $1.17$ sec.). Our end-to-end SVR model facilitates real-time head motion tracking during fMRI acquisition, ensuring reliability and robustness against uncertainties in inputs. source code, which includes the training and evaluations, will be available soon. △ Less

Submitted 6 April, 2024; originally announced April 2024.

Comments: Currently under review

CIMIL-CRC: a clinically-informed multiple instance learning framework for patient-level colorectal cancer molecular subtypes classification from H\&E stained images

Authors: Hadar Hezi, Matan Gelber, Alexander Balabanov, Yosef E. Maruvka, Moti Freiman

Abstract: Treatment approaches for colorectal cancer (CRC) are highly dependent on the molecular subtype, as immunotherapy has shown efficacy in cases with microsatellite instability (MSI) but is ineffective for the microsatellite stable (MSS) subtype. There is promising potential in utilizing deep neural networks (DNNs) to automate the differentiation of CRC subtypes by analyzing Hematoxylin and Eosin (H\&… ▽ More Treatment approaches for colorectal cancer (CRC) are highly dependent on the molecular subtype, as immunotherapy has shown efficacy in cases with microsatellite instability (MSI) but is ineffective for the microsatellite stable (MSS) subtype. There is promising potential in utilizing deep neural networks (DNNs) to automate the differentiation of CRC subtypes by analyzing Hematoxylin and Eosin (H\&E) stained whole-slide images (WSIs). Due to the extensive size of WSIs, Multiple Instance Learning (MIL) techniques are typically explored. However, existing MIL methods focus on identifying the most representative image patches for classification, which may result in the loss of critical information. Additionally, these methods often overlook clinically relevant information, like the tendency for MSI class tumors to predominantly occur on the proximal (right side) colon. We introduce `CIMIL-CRC', a DNN framework that: 1) solves the MSI/MSS MIL problem by efficiently combining a pre-trained feature extraction model with principal component analysis (PCA) to aggregate information from all patches, and 2) integrates clinical priors, particularly the tumor location within the colon, into the model to enhance patient-level classification accuracy. We assessed our CIMIL-CRC method using the average area under the curve (AUC) from a 5-fold cross-validation experimental setup for model development on the TCGA-CRC-DX cohort, contrasting it with a baseline patch-level classification, MIL-only approach, and Clinically-informed patch-level classification approach. Our CIMIL-CRC outperformed all methods (AUROC: $0.92\pm0.002$ (95\% CI 0.91-0.92), vs. $0.79\pm0.02$ (95\% CI 0.76-0.82), $0.86\pm0.01$ (95\% CI 0.85-0.88), and $0.87\pm0.01$ (95\% CI 0.86-0.88), respectively). The improvement was statistically significant. △ Less

Submitted 29 January, 2024; originally announced January 2024.

IVIM-Morph: Motion-compensated quantitative Intra-voxel Incoherent Motion (IVIM) analysis for functional fetal lung maturity assessment from diffusion-weighted MRI data

Authors: Noga Kertes, Yael Zaffrani-Reznikov, Onur Afacan, Sila Kurugol, Simon K. Warfield, Moti Freiman

Abstract: Quantitative analysis of pseudo-diffusion in diffusion-weighted magnetic resonance imaging (DWI) data shows potential for assessing fetal lung maturation and generating valuable imaging biomarkers. Yet, the clinical utility of DWI data is hindered by unavoidable fetal motion during acquisition. We present IVIM-morph, a self-supervised deep neural network model for motion-corrected quantitative ana… ▽ More Quantitative analysis of pseudo-diffusion in diffusion-weighted magnetic resonance imaging (DWI) data shows potential for assessing fetal lung maturation and generating valuable imaging biomarkers. Yet, the clinical utility of DWI data is hindered by unavoidable fetal motion during acquisition. We present IVIM-morph, a self-supervised deep neural network model for motion-corrected quantitative analysis of DWI data using the Intra-voxel Incoherent Motion (IVIM) model. IVIM-morph combines two sub-networks, a registration sub-network, and an IVIM model fitting sub-network, enabling simultaneous estimation of IVIM model parameters and motion. To promote physically plausible image registration, we introduce a biophysically informed loss function that effectively balances registration and model-fitting quality. We validated the efficacy of IVIM-morph by establishing a correlation between the predicted IVIM model parameters of the lung and gestational age (GA) using fetal DWI data of 39 subjects. IVIM-morph exhibited a notably improved correlation with gestational age (GA) when performing in-vivo quantitative analysis of fetal lung DWI data during the canalicular phase. IVIM-morph shows potential in develo** valuable biomarkers for non-invasive assessment of fetal lung maturity with DWI data. Moreover, its adaptability opens the door to potential applications in other clinical contexts where motion compensation is essential for quantitative DWI analysis. The IVIM-morph code is readily available at: https://github.com/TechnionComputationalMRILab/qDWI-Morph. △ Less

Submitted 17 January, 2024; v1 submitted 13 January, 2024; originally announced January 2024.

LUNet: Deep Learning for the Segmentation of Arterioles and Venules in High Resolution Fundus Images

Authors: Jonathan Fhima, Jan Van Eijgen, Hana Kulenovic, Valérie Debeuf, Marie Vangilbergen, Marie-Isaline Billen, Heloïse Brackenier, Moti Freiman, Ingeborg Stalmans, Joachim A. Behar

Abstract: The retina is the only part of the human body in which blood vessels can be accessed non-invasively using imaging techniques such as digital fundus images (DFI). The spatial distribution of the retinal microvasculature may change with cardiovascular diseases and thus the eyes may be regarded as a window to our hearts. Computerized segmentation of the retinal arterioles and venules (A/V) is essenti… ▽ More The retina is the only part of the human body in which blood vessels can be accessed non-invasively using imaging techniques such as digital fundus images (DFI). The spatial distribution of the retinal microvasculature may change with cardiovascular diseases and thus the eyes may be regarded as a window to our hearts. Computerized segmentation of the retinal arterioles and venules (A/V) is essential for automated microvasculature analysis. Using active learning, we created a new DFI dataset containing 240 crowd-sourced manual A/V segmentations performed by fifteen medical students and reviewed by an ophthalmologist, and developed LUNet, a novel deep learning architecture for high resolution A/V segmentation. LUNet architecture includes a double dilated convolutional block that aims to enhance the receptive field of the model and reduce its parameter count. Furthermore, LUNet has a long tail that operates at high resolution to refine the segmentation. The custom loss function emphasizes the continuity of the blood vessels. LUNet is shown to significantly outperform two state-of-the-art segmentation algorithms on the local test set as well as on four external test sets simulating distribution shifts across ethnicity, comorbidities, and annotators. We make the newly created dataset open access (upon publication). △ Less

Submitted 11 September, 2023; originally announced September 2023.

PCMC-T1: Free-breathing myocardial T1 map** with Physically-Constrained Motion Correction

Authors: Eyal Hanania, Ilya Volovik, Lilach Barkat, Israel Cohen, Moti Freiman

Abstract: T1 map** is a quantitative magnetic resonance imaging (qMRI) technique that has emerged as a valuable tool in the diagnosis of diffuse myocardial diseases. However, prevailing approaches have relied heavily on breath-hold sequences to eliminate respiratory motion artifacts. This limitation hinders accessibility and effectiveness for patients who cannot tolerate breath-holding. Image registration… ▽ More T1 map** is a quantitative magnetic resonance imaging (qMRI) technique that has emerged as a valuable tool in the diagnosis of diffuse myocardial diseases. However, prevailing approaches have relied heavily on breath-hold sequences to eliminate respiratory motion artifacts. This limitation hinders accessibility and effectiveness for patients who cannot tolerate breath-holding. Image registration can be used to enable free-breathing T1 map**. Yet, inherent intensity differences between the different time points make the registration task challenging. We introduce PCMC-T1, a physically-constrained deep-learning model for motion correction in free-breathing T1 map**. We incorporate the signal decay model into the network architecture to encourage physically-plausible deformations along the longitudinal relaxation axis. We compared PCMC-T1 to baseline deep-learning-based image registration approaches using a 5-fold experimental setup on a publicly available dataset of 210 patients. PCMC-T1 demonstrated superior model fitting quality (R2: 0.955) and achieved the highest clinical impact (clinical score: 3.93) compared to baseline methods (0.941, 0.946 and 3.34, 3.62 respectively). Anatomical alignment results were comparable (Dice score: 0.9835 vs. 0.984, 0.988). Our code and trained models are available at https://github.com/eyalhana/PCMC-T1. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: Accepted to MICCAI 2023

Exploring the Interplay Between Colorectal Cancer Subtypes Genomic Variants and Cellular Morphology: A Deep-Learning Approach

Authors: Hadar Hezi, Daniel Shats, Daniel Gurevich, Yosef E. Maruvka, Moti Freiman

Abstract: Molecular subtypes of colorectal cancer (CRC) significantly influence treatment decisions. While convolutional neural networks (CNNs) have recently been introduced for automated CRC subtype identification using H&E stained histopathological images, the correlation between CRC subtype genomic variants and their corresponding cellular morphology expressed by their imaging phenotypes is yet to be ful… ▽ More Molecular subtypes of colorectal cancer (CRC) significantly influence treatment decisions. While convolutional neural networks (CNNs) have recently been introduced for automated CRC subtype identification using H&E stained histopathological images, the correlation between CRC subtype genomic variants and their corresponding cellular morphology expressed by their imaging phenotypes is yet to be fully explored. The goal of this study was to determine such correlations by incorporating genomic variants in CNN models for CRC subtype classification from H&E images. We utilized the publicly available TCGA-CRC-DX dataset, which comprises whole slide images from 360 CRC-diagnosed patients (260 for training and 100 for testing). This dataset also provides information on CRC subtype classifications and genomic variations. We trained CNN models for CRC subtype classification that account for potential correlation between genomic variations within CRC subtypes and their corresponding cellular morphology patterns. We assessed the interplay between CRC subtypes' genomic variations and cellular morphology patterns by evaluating the CRC subtype classification accuracy of the different models in a stratified 5-fold cross-validation experimental setup using the area under the ROC curve (AUROC) and average precision (AP) as the performance metrics. Combining the CNN models account for variations in CIMP and SNP further improved classification accuracy (AUROC: 0.847$\pm$0.01 vs. 0.787$\pm$0.03, p$=$0.01, AP: 0.68$\pm$0.02 vs. 0.64$\pm$0.05). △ Less

Submitted 29 January, 2024; v1 submitted 26 March, 2023; originally announced March 2023.

P2T2: a Physically-primed deep-neural-network approach for robust $T_{2}$ distribution estimation from quantitative $T_{2}$-weighted MRI

Authors: Hadas Ben-Atya, Moti Freiman

Abstract: Estimating $T_2$ relaxation time distributions from multi-echo $T_2$-weighted MRI ($T_2W$) data can provide valuable biomarkers for assessing inflammation, demyelination, edema, and cartilage composition in various pathologies, including neurodegenerative disorders, osteoarthritis, and tumors. Deep neural network (DNN) based methods have been proposed to address the complex inverse problem of esti… ▽ More Estimating $T_2$ relaxation time distributions from multi-echo $T_2$-weighted MRI ($T_2W$) data can provide valuable biomarkers for assessing inflammation, demyelination, edema, and cartilage composition in various pathologies, including neurodegenerative disorders, osteoarthritis, and tumors. Deep neural network (DNN) based methods have been proposed to address the complex inverse problem of estimating $T_2$ distributions from MRI data, but they are not yet robust enough for clinical data with low Signal-to-Noise ratio (SNR) and are highly sensitive to distribution shifts such as variations in echo-times (TE) used during acquisition. Consequently, their application is hindered in clinical practice and large-scale multi-institutional trials with heterogeneous acquisition protocols. We propose a physically-primed DNN approach, called $P_2T_2$, that incorporates the signal decay forward model in addition to the MRI signal into the DNN architecture to improve the accuracy and robustness of $T_2$ distribution estimation. We evaluated our $P_2T_2$ model in comparison to both DNN-based methods and classical methods for $T_2$ distribution estimation using 1D and 2D numerical simulations along with clinical data. Our model improved the baseline model's accuracy for low SNR levels ($SNR<80$) which are common in the clinical setting. Further, our model achieved a $\sim$35\% improvement in robustness against distribution shifts in the acquisition process compared to previously proposed DNN models. Finally, Our $P_2T_2$ model produces the most detailed Myelin-Water fraction maps compared to baseline approaches when applied to real human MRI data. Our $P_2T_2$ model offers a reliable and precise means of estimating $T_2$ distributions from MRI data and shows promise for use in large-scale multi-institutional trials with heterogeneous acquisition protocols. △ Less

Submitted 17 May, 2023; v1 submitted 8 December, 2022; originally announced December 2022.

MA-RECON: Mask-aware deep-neural-network for robust fast MRI k-space interpolation

Authors: Nitzan Avidan, Moti Freiman

Abstract: High-quality reconstruction of MRI images from under-sampled `k-space' data, which is in the Fourier domain, is crucial for shortening MRI acquisition times and ensuring superior temporal resolution. Over recent years, a wealth of deep neural network (DNN) methods have emerged, aiming to tackle the complex, ill-posed inverse problem linked to this process. However, their instability against variat… ▽ More High-quality reconstruction of MRI images from under-sampled `k-space' data, which is in the Fourier domain, is crucial for shortening MRI acquisition times and ensuring superior temporal resolution. Over recent years, a wealth of deep neural network (DNN) methods have emerged, aiming to tackle the complex, ill-posed inverse problem linked to this process. However, their instability against variations in the acquisition process and anatomical distribution exposes a deficiency in the generalization of relevant physical models within these DNN architectures. The goal of our work is to enhance the generalization capabilities of DNN methods for k-space interpolation by introducing `MA-RECON', an innovative mask-aware DNN architecture and associated training method. Unlike preceding approaches, our `MA-RECON' architecture encodes not only the observed data but also the under-sampling mask within the model structure. It implements a tailored training approach that leverages data generated with a variety of under-sampling masks to stimulate the model's generalization of the under-sampled MRI reconstruction problem. Therefore, effectively represents the associated inverse problem, akin to the classical compressed sensing approach. The benefits of our MA-RECON approach were affirmed through rigorous testing with the widely accessible fastMRI dataset. Compared to standard DNN methods and DNNs trained with under-sampling mask augmentation, our approach demonstrated superior generalization capabilities. This resulted in a considerable improvement in robustness against variations in both the acquisition process and anatomical distribution, especially in regions with pathology. In conclusion, our mask-aware strategy holds promise for enhancing the generalization capacity and robustness of DNN-based methodologies for MRI reconstruction from undersampled k-space data. △ Less

Submitted 26 November, 2023; v1 submitted 31 August, 2022; originally announced September 2022.

Comments: Accepted to Computer Methods and Programs in Biomedicine

Patient-level Microsatellite Stability Assessment from Whole Slide Images By Combining Momentum Contrast Learning and Group Patch Embeddings

Authors: Daniel Shats, Hadar Hezi, Guy Shani, Yosef E. Maruvka, Moti Freiman

Abstract: Assessing microsatellite stability status of a patient's colorectal cancer is crucial in personalizing treatment regime. Recently, convolutional-neural-networks (CNN) combined with transfer-learning approaches were proposed to circumvent traditional laboratory testing for determining microsatellite status from hematoxylin and eosin stained biopsy whole slide images (WSI). However, the high resolut… ▽ More Assessing microsatellite stability status of a patient's colorectal cancer is crucial in personalizing treatment regime. Recently, convolutional-neural-networks (CNN) combined with transfer-learning approaches were proposed to circumvent traditional laboratory testing for determining microsatellite status from hematoxylin and eosin stained biopsy whole slide images (WSI). However, the high resolution of WSI practically prevent direct classification of the entire WSI. Current approaches bypass the WSI high resolution by first classifying small patches extracted from the WSI, and then aggregating patch-level classification logits to deduce the patient-level status. Such approaches limit the capacity to capture important information which resides at the high resolution WSI data. We introduce an effective approach to leverage WSI high resolution information by momentum contrastive learning of patch embeddings along with training a patient-level classifier on groups of those embeddings. Our approach achieves up to 7.4\% better accuracy compared to the straightforward patch-level classification and patient level aggregation approach with a higher stability (AUC, $0.91 \pm 0.01$ vs. $0.85 \pm 0.04$, p-value$<0.01$). Our code can be found at https://github.com/TechnionComputationalMRILab/colorectal_cancer_ai. △ Less

Submitted 22 August, 2022; originally announced August 2022.

Comments: To appear in the proceedings of the ECCV workshop on Medical Computer Vision (ECCV-MCV 2022). Link: https://mcv-workshop.github.io/

Lirot.ai: A Novel Platform for Crowd-Sourcing Retinal Image Segmentations

Authors: Jonathan Fhima, Jan Van Eijgen, Moti Freiman, Ingeborg Stalmans, Joachim A. Behar

Abstract: Introduction: For supervised deep learning (DL) tasks, researchers need a large annotated dataset. In medical data science, one of the major limitations to develop DL models is the lack of annotated examples in large quantity. This is most often due to the time and expertise required to annotate. We introduce Lirot. ai, a novel platform for facilitating and crowd-sourcing image segmentations. Meth… ▽ More Introduction: For supervised deep learning (DL) tasks, researchers need a large annotated dataset. In medical data science, one of the major limitations to develop DL models is the lack of annotated examples in large quantity. This is most often due to the time and expertise required to annotate. We introduce Lirot. ai, a novel platform for facilitating and crowd-sourcing image segmentations. Methods: Lirot. ai is composed of three components; an iPadOS client application named Lirot. ai-app, a backend server named Lirot. ai-server and a python API name Lirot. ai-API. Lirot. ai-app was developed in Swift 5.6 and Lirot. ai-server is a firebase backend. Lirot. ai-API allows the management of the database. Lirot. ai-app can be installed on as many iPadOS devices as needed so that annotators may be able to perform their segmentation simultaneously and remotely. We incorporate Apple Pencil compatibility, making the segmentation faster, more accurate, and more intuitive for the expert than any other computer-based alternative. Results: We demonstrate the usage of Lirot. ai for the creation of a retinal fundus dataset with reference vasculature segmentations. Discussion and future work: We will use active learning strategies to continue enlarging our retinal fundus dataset by including a more efficient process to select the images to be annotated and distribute them to annotators. △ Less

Submitted 14 January, 2024; v1 submitted 22 August, 2022; originally announced August 2022.

qDWI-Morph: Motion-compensated quantitative Diffusion-Weighted MRI analysis for fetal lung maturity assessment

Authors: Yael Zaffrani-Reznikov, Onur Afacan, Sila Kurugol, Simon Warfield, Moti Freiman

Abstract: Quantitative analysis of fetal lung Diffusion-Weighted MRI (DWI) data shows potential in providing quantitative imaging biomarkers that indirectly reflect fetal lung maturation. However, fetal motion during the acquisition hampered quantitative analysis of the acquired DWI data and, consequently, reliable clinical utilization. We introduce qDWI-morph, an unsupervised deep-neural-network architectu… ▽ More Quantitative analysis of fetal lung Diffusion-Weighted MRI (DWI) data shows potential in providing quantitative imaging biomarkers that indirectly reflect fetal lung maturation. However, fetal motion during the acquisition hampered quantitative analysis of the acquired DWI data and, consequently, reliable clinical utilization. We introduce qDWI-morph, an unsupervised deep-neural-network architecture for motion compensated quantitative DWI (qDWI) analysis. Our approach couples a registration sub-network with a quantitative DWI model fitting sub-network. We simultaneously estimate the qDWI parameters and the motion model by minimizing a bio-physically-informed loss function integrating a registration loss and a model fitting quality loss. We demonstrated the added-value of qDWI-morph over: 1) a baseline qDWI analysis without motion compensation and 2) a baseline deep-learning model incorporating registration loss solely. The qDWI-morph achieved a substantially improved correlation with the gestational age through in-vivo qDWI analysis of fetal lung DWI data (R-squared=0.32 vs. 0.13, 0.28). Our qDWI-morph has the potential to enable motion-compensated quantitative analysis of DWI data and to provide clinically feasible bio-markers for non-invasive fetal lung maturity assessment. Our code is available at: https://github.com/TechnionComputationalMRILab/qDWI-Morph. △ Less

Submitted 21 August, 2022; originally announced August 2022.

Comments: Accepted to ECCV-MCV: https://mcv-workshop.github.io/

NPB-REC: Non-parametric Assessment of Uncertainty in Deep-learning-based MRI Reconstruction from Undersampled Data

Authors: Samah Khawaled, Moti Freiman

Abstract: Uncertainty quantification in deep-learning (DL) based image reconstruction models is critical for reliable clinical decision making based on the reconstructed images. We introduce "NPB-REC", a non-parametric fully Bayesian framework for uncertainty assessment in MRI reconstruction from undersampled "k-space" data. We use Stochastic gradient Langevin dynamics (SGLD) during the training phase to ch… ▽ More Uncertainty quantification in deep-learning (DL) based image reconstruction models is critical for reliable clinical decision making based on the reconstructed images. We introduce "NPB-REC", a non-parametric fully Bayesian framework for uncertainty assessment in MRI reconstruction from undersampled "k-space" data. We use Stochastic gradient Langevin dynamics (SGLD) during the training phase to characterize the posterior distribution of the network weights. We demonstrated the added-value of our approach on the multi-coil brain MRI dataset, from the fastmri challenge, in comparison to the baseline E2E-VarNet with and without inference-time dropout. Our experiments show that NPB-REC outperforms the baseline by means of reconstruction accuracy (PSNR and SSIM of $34.55$, $0.908$ vs. $33.08$, $0.897$, $p<0.01$) in high acceleration rates ($R=8$). This is also measured in regions of clinical annotations. More significantly, it provides a more accurate estimate of the uncertainty that correlates with the reconstruction error, compared to the Monte-Carlo inference time Dropout method (Pearson correlation coefficient of $R=0.94$ vs. $R=0.91$). The proposed approach has the potential to facilitate safe utilization of DL based methods for MRI reconstruction from undersampled data. Code and trained models are available in \url{https://github.com/samahkh/NPB-REC}. △ Less

Submitted 8 August, 2022; originally announced August 2022.

Comments: To appear in the proceedings of the International Workshop on Machine Learning for Medical Image Reconstruction (MLMIR 2022), https://sites.google.com/view/mlmir2022

PVBM: A Python Vasculature Biomarker Toolbox Based On Retinal Blood Vessel Segmentation

Authors: Jonathan Fhima, Jan Van Eijgen, Ingeborg Stalmans, Yevgeniy Men, Moti Freiman, Joachim A. Behar

Abstract: Introduction: Blood vessels can be non-invasively visualized from a digital fundus image (DFI). Several studies have shown an association between cardiovascular risk and vascular features obtained from DFI. Recent advances in computer vision and image segmentation enable automatising DFI blood vessel segmentation. There is a need for a resource that can automatically compute digital vasculature bi… ▽ More Introduction: Blood vessels can be non-invasively visualized from a digital fundus image (DFI). Several studies have shown an association between cardiovascular risk and vascular features obtained from DFI. Recent advances in computer vision and image segmentation enable automatising DFI blood vessel segmentation. There is a need for a resource that can automatically compute digital vasculature biomarkers (VBM) from these segmented DFI. Methods: In this paper, we introduce a Python Vasculature BioMarker toolbox, denoted PVBM. A total of 11 VBMs were implemented. In particular, we introduce new algorithmic methods to estimate tortuosity and branching angles. Using PVBM, and as a proof of usability, we analyze geometric vascular differences between glaucomatous patients and healthy controls. Results: We built a fully automated vasculature biomarker toolbox based on DFI segmentations and provided a proof of usability to characterize the vascular changes in glaucoma. For arterioles and venules, all biomarkers were significant and lower in glaucoma patients compared to healthy controls except for tortuosity, venular singularity length and venular branching angles. Conclusion: We have automated the computation of 11 VBMs from retinal blood vessel segmentation. The PVBM toolbox is made open source under a GNU GPL 3 license and is available on physiozoo.com (following publication). △ Less

Submitted 31 July, 2022; originally announced August 2022.

PD-DWI: Predicting response to neoadjuvant chemotherapy in invasive breast cancer with Physiologically-Decomposed Diffusion-Weighted MRI machine-learning model

Authors: Maya Gilad, Moti Freiman

Abstract: Early prediction of pathological complete response (pCR) following neoadjuvant chemotherapy (NAC) for breast cancer plays a critical role in surgical planning and optimizing treatment strategies. Recently, machine and deep-learning based methods were suggested for early pCR prediction from multi-parametric MRI (mp-MRI) data including dynamic contrast-enhanced MRI and diffusion-weighted MRI (DWI) w… ▽ More Early prediction of pathological complete response (pCR) following neoadjuvant chemotherapy (NAC) for breast cancer plays a critical role in surgical planning and optimizing treatment strategies. Recently, machine and deep-learning based methods were suggested for early pCR prediction from multi-parametric MRI (mp-MRI) data including dynamic contrast-enhanced MRI and diffusion-weighted MRI (DWI) with moderate success. We introduce PD-DWI, a physiologically decomposed DWI machine-learning model to predict pCR from DWI and clinical data. Our model first decomposes the raw DWI data into the various physiological cues that are influencing the DWI signal and then uses the decomposed data, in addition to clinical variables, as the input features of a radiomics-based XGBoost model. We demonstrated the added-value of our PD-DWI model over conventional machine-learning approaches for pCR prediction from mp-MRI data using the publicly available Breast Multi-parametric MRI for prediction of NAC Response (BMMR2) challenge. Our model substantially improves the area under the curve (AUC), compared to the current best result on the leaderboard (0.8849 vs. 0.8397) for the challenge test set. PD-DWI has the potential to improve prediction of pCR following NAC for breast cancer, reduce overall mp-MRI acquisition times and eliminate the need for contrast-agent injection. △ Less

Submitted 12 June, 2022; originally announced June 2022.

Comments: Accepted to Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 to be held during Sept 18-22 in Singapore

SUPER-IVIM-DC: Intra-voxel incoherent motion based Fetal lung maturity assessment from limited DWI data using supervised learning coupled with data-consistency

Authors: Noam Korngut, Elad Rotman, Onur Afacan, Sila Kurugol, Yael Zaffrani-Reznikov, Shira Nemirovsky-Rotman, Simon Warfield, Moti Freiman

Abstract: Intra-voxel incoherent motion (IVIM) analysis of fetal lungs Diffusion-Weighted MRI (DWI) data shows potential in providing quantitative imaging bio-markers that reflect, indirectly, diffusion and pseudo-diffusion for non-invasive fetal lung maturation assessment. However, long acquisition times, due to the large number of different ``b-value'' images required for IVIM analysis, precluded clinical… ▽ More Intra-voxel incoherent motion (IVIM) analysis of fetal lungs Diffusion-Weighted MRI (DWI) data shows potential in providing quantitative imaging bio-markers that reflect, indirectly, diffusion and pseudo-diffusion for non-invasive fetal lung maturation assessment. However, long acquisition times, due to the large number of different ``b-value'' images required for IVIM analysis, precluded clinical feasibility. We introduce SUPER-IVIM-DC a deep-neural-networks (DNN) approach which couples supervised loss with a data-consistency term to enable IVIM analysis of DWI data acquired with a limited number of b-values. We demonstrated the added-value of SUPER-IVIM-DC over both classical and recent DNN approaches for IVIM analysis through numerical simulations, healthy volunteer study, and IVIM analysis of fetal lung maturation from fetal DWI data. Our numerical simulations and healthy volunteer study show that SUPER-IVIM-DC estimates of the IVIM model parameters from limited DWI data had lower normalized root mean-squared error compared to previous DNN-based approaches. Further, SUPER-IVIM-DC estimates of the pseudo-diffusion fraction parameter from limited DWI data of fetal lungs correlate better with gestational age compared to both to classical and DNN-based approaches (0.555 vs. 0.463 and 0.310). SUPER-IVIM-DC has the potential to reduce the long acquisition times associated with IVIM analysis of DWI data and to provide clinically feasible bio-markers for non-invasive fetal lung maturity assessment. △ Less

Submitted 30 June, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

Comments: Accepted to the International Conference on Medical Image Computing and Computer Assisted Intervention - MICCAI 2022, to be held during Sept 18-22 in Singapore

Image translation of Ultrasound to Pseudo Anatomical Display by CycleGAN

Authors: Lilach Barkat, Moti Freiman, Haim Azhari

Abstract: Ultrasound is the second most used modality in medical imaging. It is cost effective, hazardless, portable and implemented routinely in numerous clinical procedures. Nonetheless, image quality is characterized by granulated appearance, poor SNR and speckle noise. Specific for malignant tumors, the margins are blurred and indistinct. Thus, there is a great need for improving ultrasound image qualit… ▽ More Ultrasound is the second most used modality in medical imaging. It is cost effective, hazardless, portable and implemented routinely in numerous clinical procedures. Nonetheless, image quality is characterized by granulated appearance, poor SNR and speckle noise. Specific for malignant tumors, the margins are blurred and indistinct. Thus, there is a great need for improving ultrasound image quality. We hypothesize that this can be achieved, using neural networks, by translation into a more realistic display which mimics an anatomical cut through the tissue. In order to achieve this goal, the preferable approach would be to use a set of paired images. However, this is practically impossible in our case. Therefore, Cycle Generative Adversarial Network (CycleGAN) was used, in order to learn each domain properties separately and enforce cross domain cycle consistency. The two datasets which were used for training the model were "Breast Ultrasound Images" (BUSI) and a set of optic images of poultry breast tissue samples acquired at our lab. The generated pseudo anatomical images provide improved visual discrimination of the lesions with clearer border definition and pronounced contrast. In order to evaluate the preservation of the anatomical features, the lesions in the ultrasonic images and the generated pseudo anatomical images were both automatically segmented and compared. This comparison yielded median dice score of 0.91 for the benign tumors and 0.70 for the malignant ones. The median lesion center error was 0.58% and 3.27% for the benign and malignancies respectively and the median area error index was 0.40% and 4.34% for the benign and malignancies respectively. In conclusion, these generated pseudo anatomical images, which are presented in a more intuitive way, enhance tissue anatomy and can potentially simplify the diagnosis and improve the clinical outcome. △ Less

Submitted 10 June, 2022; v1 submitted 16 February, 2022; originally announced February 2022.

Comments: 9 pages, 5 figures

Non Parametric Data Augmentations Improve Deep-Learning based Brain Tumor Segmentation

Authors: Hadas Ben-Atya, Ori Rajchert, Liran Goshen, Moti Freiman

Abstract: Automatic brain tumor segmentation from Magnetic Resonance Imaging (MRI) data plays an important role in assessing tumor response to therapy and personalized treatment stratification.Manual segmentation is tedious and subjective.Deep-learning-based algorithms for brain tumor segmentation have the potential to provide objective and fast tumor segmentation.However, the training of such algorithms re… ▽ More Automatic brain tumor segmentation from Magnetic Resonance Imaging (MRI) data plays an important role in assessing tumor response to therapy and personalized treatment stratification.Manual segmentation is tedious and subjective.Deep-learning-based algorithms for brain tumor segmentation have the potential to provide objective and fast tumor segmentation.However, the training of such algorithms requires large datasets which are not always available. Data augmentation techniques may reduce the need for large datasets.However current approaches are mostly parametric and may result in suboptimal performance.We introduce two non-parametric methods of data augmentation for brain tumor segmentation: the mixed structure regularization (MSR) and shuffle pixels noise (SPN).We evaluated the added value of the MSR and SPN augmentation on the brain tumor segmentation (BraTS) 2018 challenge dataset with the encoder-decoder nnU-Net architecture as the segmentation algorithm.Both MSR and SPN improve the nnU-Net segmentation accuracy compared to parametric Gaussian noise augmentation.Mean dice score increased from 80% to 82% and p-values=0.0022, 0.0028 when comparing MSR to non-parametric augmentation for the tumor core and whole tumor experiments respectively.The proposed MSR and SPN augmentations have the potential to improve neural-networks performance in other tasks as well. △ Less

Submitted 25 November, 2021; originally announced November 2021.

Journal ref: 2021 IEEE COMCAS

NPBDREG: Uncertainty Assessment in Diffeomorphic Brain MRI Registration using a Non-parametric Bayesian Deep-Learning Based Approach

Authors: Samah Khawaled, Moti Freiman

Abstract: Quantification of uncertainty in deep-neural-networks (DNN) based image registration algorithms plays a critical role in the deployment of image registration algorithms for clinical applications such as surgical planning, intraoperative guidance, and longitudinal monitoring of disease progression or treatment efficacy as well as in research-oriented processing pipelines. Currently available approa… ▽ More Quantification of uncertainty in deep-neural-networks (DNN) based image registration algorithms plays a critical role in the deployment of image registration algorithms for clinical applications such as surgical planning, intraoperative guidance, and longitudinal monitoring of disease progression or treatment efficacy as well as in research-oriented processing pipelines. Currently available approaches for uncertainty estimation in DNN-based image registration algorithms may result in sub-optimal clinical decision making due to potentially inaccurate estimation of the uncertainty of the registration stems for the assumed parametric distribution of the registration latent space. We introduce NPBDREG, a fully non-parametric Bayesian framework for uncertainty estimation in DNN-based deformable image registration by combining an Adam optimizer with stochastic gradient Langevin dynamics (SGLD) to characterize the underlying posterior distribution through posterior sampling. Thus, it has the potential to provide uncertainty estimates that are highly correlated with the presence of out of distribution data. We demonstrated the added-value of NPBDREG, compared to the baseline probabilistic VoxelMorph model (PrVXM), on brain MRI image registration using $390$ image pairs from four publicly available databases: MGH10, CMUC12, ISBR18 and LPBA40. The NPBDREG shows a better correlation of the predicted uncertainty with out-of-distribution data ($r>0.95$ vs. $r<0.5$) as well as a 7.3%improvement in the registration accuracy (Dice score, $0.74$ vs. $0.69$, $p \ll 0.01$), and 18% improvement in registration smoothness (percentage of folds in the deformation field, 0.014 vs. 0.017, $p \ll 0.01$). Finally, NPBDREG demonstrated a better generalization capability for data corrupted by a mixed structure noise (Dice score of $0.73$ vs. $0.69$, $p \ll 0.01$) compared to the baseline PrVXM approach. △ Less

Submitted 9 June, 2022; v1 submitted 15 August, 2021; originally announced August 2021.

Comments: To appear in Computerized Medical Imaging and Graphics, DOI: https://doi.org/10.1016/j.compmedimag.2022.102087. This is an extension representing a more comprehensive work extending preliminary work presented at arXiv:2008.03949

Unsupervised Deep-Learning Based Deformable Image Registration: A Bayesian Framework

Authors: Samah Khawaled, Moti Freiman

Abstract: Unsupervised deep-learning (DL) models were recently proposed for deformable image registration tasks. In such models, a neural-network is trained to predict the best deformation field by minimizing some dissimilarity function between the moving and the target images. After training on a dataset without reference deformation fields available, such a model can be used to rapidly predict the deforma… ▽ More Unsupervised deep-learning (DL) models were recently proposed for deformable image registration tasks. In such models, a neural-network is trained to predict the best deformation field by minimizing some dissimilarity function between the moving and the target images. After training on a dataset without reference deformation fields available, such a model can be used to rapidly predict the deformation field between newly seen moving and target images. Currently, the training process effectively provides a point-estimate of the network weights rather than characterizing their entire posterior distribution. This may result in a potential over-fitting which may yield sub-optimal results at inference phase, especially for small-size datasets, frequently present in the medical imaging domain. We introduce a fully Bayesian framework for unsupervised DL-based deformable image registration. Our method provides a principled way to characterize the true posterior distribution, thus, avoiding potential over-fitting. We used stochastic gradient Langevin dynamics (SGLD) to conduct the posterior sampling, which is both theoretically well-founded and computationally efficient. We demonstrated the added-value of our Basyesian unsupervised DL-based registration framework on the MNIST and brain MRI (MGH10) datasets in comparison to the VoxelMorph unsupervised DL-based image registration framework. Our experiments show that our approach provided better estimates of the deformation field by means of improved mean-squared-error ($0.0063$ vs. $0.0065$) and Dice coefficient ($0.73$ vs. $0.71$) for the MNIST and the MGH10 datasets respectively. Further, our approach provides an estimate of the uncertainty in the deformation-field by characterizing the true posterior distribution. △ Less

Submitted 10 August, 2020; originally announced August 2020.

Comments: 10 pages

Learning a sparse database for patch-based medical image segmentation

Authors: Moti Freiman, Hannes Nickisch, Holger Schmitt, Pal Maurovich-Horvat, Patrick Donnelly, Mani Vembar, Liran Goshen

Abstract: We introduce a functional for the learning of an optimal database for patch-based image segmentation with application to coronary lumen segmentation from coronary computed tomography angiography (CCTA) data. The proposed functional consists of fidelity, sparseness and robustness to small-variations terms and their associated weights. Existing work address database optimization by prototype selecti… ▽ More We introduce a functional for the learning of an optimal database for patch-based image segmentation with application to coronary lumen segmentation from coronary computed tomography angiography (CCTA) data. The proposed functional consists of fidelity, sparseness and robustness to small-variations terms and their associated weights. Existing work address database optimization by prototype selection aiming to optimize the database by either adding or removing prototypes according to a set of predefined rules. In contrast, we formulate the database optimization task as an energy minimization problem that can be solved using standard numerical tools. We apply the proposed database optimization functional to the task of optimizing a database for patch-base coronary lumen segmentation. Our experiments using the publicly available MICCAI 2012 coronary lumen segmentation challenge data show that optimizing the database using the proposed approach reduced database size by 96% while maintaining the same level of lumen segmentation accuracy. Moreover, we show that the optimized database yields an improved specificity of CCTA based fractional flow reserve (0.73 vs 0.7 for all lesions and 0.68 vs 0.65 for obstructive lesions) using a training set of 132 (76 obstructive) coronary lesions with invasively measured FFR as the reference. △ Less

Submitted 25 June, 2019; originally announced June 2019.

Journal ref: Wu G., Munsell B., Zhan Y., Bai W., Sanroma G., Coupé P. (eds) Patch-Based Techniques in Medical Imaging. Patch-MI 2017. Lecture Notes in Computer Science, vol 10530. Springer, Cham

Improving CCTA based lesions' hemodynamic significance assessment by accounting for partial volume modeling in automatic coronary lumen segmentation

Authors: Moti Freiman, Hannes Nickisch, Sven Prevrhal, Holger Schmitt, Mani Vembar, Pál Maurovich-Horvat, Patrick Donnelly, Liran Goshen

Abstract: Purpose: The goal of this study was to assess the potential added benefit of accounting for partial volume effects (PVE) in an automatic coronary lumen segmentation algorithm from coronary computed tomography angiography (CCTA). Materials and methods: We assessed the potential added value of PVE integration as a part of the automatic coronary lumen segmentation algorithm by means of segmentation a… ▽ More Purpose: The goal of this study was to assess the potential added benefit of accounting for partial volume effects (PVE) in an automatic coronary lumen segmentation algorithm from coronary computed tomography angiography (CCTA). Materials and methods: We assessed the potential added value of PVE integration as a part of the automatic coronary lumen segmentation algorithm by means of segmentation accuracy using the MICCAI 2012 challenge framework and by means of flow simulation overall accuracy, sensitivity, specificity, negative and positive predictive values and the receiver operated characteristic (ROC) area under the curve. We also evaluated the potential benefit of accounting for PVE in automatic segmentation for flow-simulation for lesions that were diagnosed as obstructive based on CCTA, which could have indicated a need for an invasive exam and revascularization. Results: Our segmentation algorithm improves the maximal surface distance error by ~39% compared to previously published method on the 18 datasets 50 from the MICCAI 2012 challenge with comparable Dice and mean surface distance. Results with and without accounting for PVE were comparable. In contrast, integrating PVE analysis into an automatic coronary lumen segmentation algorithm improved the flow simulation specificity from 0.6 to 0.68 with the same sensitivity of 0.83. Also, accounting for PVE improved the area under the ROC curve for detecting hemodynamically significant CAD from 0.76 to 0.8 compared to automatic segmentation without PVE analysis with invasive FFR threshold of 0.8 as the reference standard. The improvement in the AUC was statistically significant (N=76, Delong's test, p=0.012). Conclusion: Accounting for the partial volume effects in automatic coronary lumen segmentation algorithms has the potential to improve the accuracy of CCTA-based hemodynamic assessment of coronary artery lesions. △ Less

Submitted 24 June, 2019; originally announced June 2019.

Journal ref: Medical Physics 44: 1040-1049, 2017

Unsupervised Abnormality Detection through Mixed Structure Regularization (MSR) in Deep Sparse Autoencoders

Authors: Moti Freiman, Ravindra Manjeshwar, Liran Goshen

Abstract: Deep sparse auto-encoders with mixed structure regularization (MSR) in addition to explicit sparsity regularization term and stochastic corruption of the input data with Gaussian noise have the potential to improve unsupervised abnormality detection. Unsupervised abnormality detection based on identifying outliers using deep sparse auto-encoders is a very appealing approach for medical computer ai… ▽ More Deep sparse auto-encoders with mixed structure regularization (MSR) in addition to explicit sparsity regularization term and stochastic corruption of the input data with Gaussian noise have the potential to improve unsupervised abnormality detection. Unsupervised abnormality detection based on identifying outliers using deep sparse auto-encoders is a very appealing approach for medical computer aided detection systems as it requires only healthy data for training rather than expert annotated abnormality. In the task of detecting coronary artery disease from Coronary Computed Tomography Angiography (CCTA), our results suggests that the MSR has the potential to improve overall performance by 20-30% compared to deep sparse and denoising auto-encoders. △ Less

Submitted 28 February, 2019; originally announced February 2019.

Comments: Accepted for publication in the journal: "Medical Physics" (2019)