-
Segmentation-Free Outcome Prediction in Head and Neck Cancer: Deep Learning-based Feature Extraction from Multi-Angle Maximum Intensity Projections (MA-MIPs) of PET Images
Authors:
Amirhosein Toosi,
Isaac Shiri,
Habib Zaidi,
Arman Rahmim
Abstract:
We introduce an innovative, simple, effective segmentation-free approach for outcome prediction in head \& neck cancer (HNC) patients. By harnessing deep learning-based feature extraction techniques and multi-angle maximum intensity projections (MA-MIPs) applied to Fluorodeoxyglucose Positron Emission Tomography (FDG-PET) volumes, our proposed method eliminates the need for manual segmentations of…
▽ More
We introduce an innovative, simple, effective segmentation-free approach for outcome prediction in head \& neck cancer (HNC) patients. By harnessing deep learning-based feature extraction techniques and multi-angle maximum intensity projections (MA-MIPs) applied to Fluorodeoxyglucose Positron Emission Tomography (FDG-PET) volumes, our proposed method eliminates the need for manual segmentations of regions-of-interest (ROIs) such as primary tumors and involved lymph nodes. Instead, a state-of-the-art object detection model is trained to perform automatic crop** of the head and neck region on the PET volumes. A pre-trained deep convolutional neural network backbone is then utilized to extract deep features from MA-MIPs obtained from 72 multi-angel axial rotations of the cropped PET volumes. These deep features extracted from multiple projection views of the PET volumes are then aggregated and fused, and employed to perform recurrence-free survival analysis on a cohort of 489 HNC patients. The proposed approach outperforms the best performing method on the target dataset for the task of recurrence-free survival analysis. By circumventing the manual delineation of the malignancies on the FDG PET-CT images, our approach eliminates the dependency on subjective interpretations and highly enhances the reproducibility of the proposed survival analysis method.
△ Less
Submitted 2 May, 2024;
originally announced May 2024.
-
Semi-supervised learning towards automated segmentation of PET images with limited annotations: Application to lymphoma patients
Authors:
Fereshteh Yousefirizi,
Isaac Shiri,
Joo Hyun O,
Ingrid Bloise,
Patrick Martineau,
Don Wilson,
François Bénard,
Laurie H. Sehn,
Kerry J. Savage,
Habib Zaidi,
Carlos F. Uribe,
Arman Rahmim
Abstract:
The time-consuming task of manual segmentation challenges routine systematic quantification of disease burden. Convolutional neural networks (CNNs) hold significant promise to reliably identify locations and boundaries of tumors from PET scans. We aimed to leverage the need for annotated data via semi-supervised approaches, with application to PET images of diffuse large B-cell lymphoma (DLBCL) an…
▽ More
The time-consuming task of manual segmentation challenges routine systematic quantification of disease burden. Convolutional neural networks (CNNs) hold significant promise to reliably identify locations and boundaries of tumors from PET scans. We aimed to leverage the need for annotated data via semi-supervised approaches, with application to PET images of diffuse large B-cell lymphoma (DLBCL) and primary mediastinal large B-cell lymphoma (PMBCL). We analyzed 18F-FDG PET images of 292 patients with PMBCL (n=104) and DLBCL (n=188) (n=232 for training and validation, and n=60 for external testing). We employed FCM and MS losses for training a 3D U-Net with different levels of supervision: i) fully supervised methods with labeled FCM (LFCM) as well as Unified focal and Dice loss functions, ii) unsupervised methods with Robust FCM (RFCM) and Mumford-Shah (MS) loss functions, and iii) Semi-supervised methods based on FCM (RFCM+LFCM), as well as MS loss in combination with supervised Dice loss (MS+Dice). Unified loss function yielded higher Dice score (mean +/- standard deviation (SD)) (0.73 +/- 0.03; 95% CI, 0.67-0.8) compared to Dice loss (p-value<0.01). Semi-supervised (RFCM+alpha*LFCM) with alpha=0.3 showed the best performance, with a Dice score of 0.69 +/- 0.03 (95% CI, 0.45-0.77) outperforming (MS+alpha*Dice) for any supervision level (any alpha) (p<0.01). The best performer among (MS+alpha*Dice) semi-supervised approaches with alpha=0.2 showed a Dice score of 0.60 +/- 0.08 (95% CI, 0.44-0.76) compared to another supervision level in this semi-supervised approach (p<0.01). Semi-supervised learning via FCM loss (RFCM+alpha*LFCM) showed improved performance compared to supervised approaches. Considering the time-consuming nature of expert manual delineations and intra-observer variabilities, semi-supervised approaches have significant potential for automated segmentation workflows.
△ Less
Submitted 25 March, 2024; v1 submitted 19 December, 2022;
originally announced December 2022.
-
Tensor Radiomics: Paradigm for Systematic Incorporation of Multi-Flavoured Radiomics Features
Authors:
Arman Rahmim,
Amirhosein Toosi,
Mohammad R. Salmanpour,
Natalia Dubljevic,
Ian Janzen,
Isaac Shiri,
Ren Yuan,
Cheryl Ho,
Habib Zaidi,
Calum MacAulay,
Carlos Uribe,
Fereshteh Yousefirizi
Abstract:
Radiomics features extract quantitative information from medical images, towards the derivation of biomarkers for clinical tasks, such as diagnosis, prognosis, or treatment response assessment. Different image discretization parameters (e.g. bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels can be used to generate radiomics features and ultimate…
▽ More
Radiomics features extract quantitative information from medical images, towards the derivation of biomarkers for clinical tasks, such as diagnosis, prognosis, or treatment response assessment. Different image discretization parameters (e.g. bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels can be used to generate radiomics features and ultimately signatures. Commonly, only one set of parameters is used; resulting in only one value or flavour for a given RF. We propose tensor radiomics (TR) where tensors of features calculated with multiple combinations of parameters (i.e. flavours) are utilized to optimize the construction of radiomics signatures. We present examples of TR as applied to PET/CT, MRI, and CT imaging invoking machine learning or deep learning solutions, and reproducibility analyses: (1) TR via varying bin sizes on CT images of lung cancer and PET-CT images of head & neck cancer (HNC) for overall survival prediction. A hybrid deep neural network, referred to as TR-Net, along with two ML-based flavour fusion methods showed improved accuracy compared to regular rediomics features. (2) TR built from different segmentation perturbations and different bin sizes for classification of late-stage lung cancer response to first-line immunotherapy using CT images. TR improved predicted patient responses. (3) TR via multi-flavour generated radiomics features in MR imaging showed improved reproducibility when compared to many single-flavour features. (4) TR via multiple PET/CT fusions in HNC. Flavours were built from different fusions using methods, such as Laplacian pyramids and wavelet transforms. TR improved overall survival prediction. Our results suggest that the proposed TR paradigm has the potential to improve performance capabilities in different medical imaging tasks.
△ Less
Submitted 24 October, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Non-Invasive Fuhrman Grading of Clear Cell Renal Cell Carcinoma Using Computed Tomography Radiomics Features and Machine Learning
Authors:
Mostafa Nazari,
Isaac Shiri,
Ghasem Hajianfar,
Niki Oveisi,
Hamid Abdollahi,
Mohammad Reza Deevband,
Mehrdad Oveisi
Abstract:
Purpose: To identify optimal classification methods for computed tomography (CT) radiomics-based preoperative prediction of clear cells renal cell carcinoma (ccRCC) grade. Methods and material: Seventy one ccRCC patients were included in the study. Three image preprocessing techniques (Laplacian of Gaussian, wavelet filter, and discretization of the intensity values) were applied on tumor volumes.…
▽ More
Purpose: To identify optimal classification methods for computed tomography (CT) radiomics-based preoperative prediction of clear cells renal cell carcinoma (ccRCC) grade. Methods and material: Seventy one ccRCC patients were included in the study. Three image preprocessing techniques (Laplacian of Gaussian, wavelet filter, and discretization of the intensity values) were applied on tumor volumes. In total, 2530 radiomics features (tumor shape and size, intensity statistics, and texture) were extracted from each segmented tumor volume. Univariate analysis was performed to assess the association of each feature with the histological condition. In the case of multivariate analysis, the following was implemented: three feature selection including the least absolute shrinkage and selection operator (LASSO), students t-test and minimum Redundancy Maximum Relevance (mRMR) algorithms. These selected features were then used to construct three classification models (SVM, random forest, and logistic regression) to discriminate the high from low-grade ccRCC at nephrectomy. Lastly, multivariate model performance was evaluated on the bootstrapped validation cohort using the area under receiver operating characteristic curve (AUC). Results: Univariate analysis demonstrated that among different image sets, 128 bin discretized images have statistically significant different (q-value < 0.05) texture parameters with a mean of AUC 0.74 (q-value < 0.05). The three ML-based classifier shows proficient discrimination of the high from low-grade ccRCC. The AUC was 0.78 in logistic regression, 0.62 in random forest, and 0.83 in SVM model, respectively. Conclusion: Radiomics features can be a useful and promising non-invasive method for preoperative evaluation of ccRCC Fuhrman grades. Key words: RCC, Radiomics, Machine Learning, Computed Tomography
△ Less
Submitted 26 September, 2019;
originally announced September 2019.
-
Cardiac SPECT Radiomics Features Repeatability and Reproducibility: A Multi Scanner Phantom Study
Authors:
Mohammad Edalat-Javid,
Isaac Shiri,
Ghasem Hajianfar,
Hamid Abdollahi,
Niki Oveisi,
Mohammad Javadian,
Mojtaba Shamsaei Zafarghandi,
Hadi Malek,
Ahmad Bitarafan-Rajabi,
Mehrdad Oveisi
Abstract:
Background: The aim of this study was to assess the robustness of cardiac SPECT radiomics features against changes in imaging settings including acquisition and reconstruction settings.
Methods: Four scanners were used to acquire SPECT scans of a cardiac phantom with 5mCi of 99mTc. The effects of different image acquisition and reconstruction settings including the Number of View, View Matrix Si…
▽ More
Background: The aim of this study was to assess the robustness of cardiac SPECT radiomics features against changes in imaging settings including acquisition and reconstruction settings.
Methods: Four scanners were used to acquire SPECT scans of a cardiac phantom with 5mCi of 99mTc. The effects of different image acquisition and reconstruction settings including the Number of View, View Matrix Size, attenuation correction, image reconstruction algorithm, number of iterations, number of subsets, type of filter, full width at half maximum (FWHM) of Gaussian filter, Butterworth filter order, and Butterworth filter cut-off were studied. In total 5263 different images were reconstructed. Eighty-seven radiomic features including first, second, and high order textures were extracted from images. To assess reproducibility and repeatability the coefficient of variation (COV) was used for each image feature over the different imaging settings.
Result: IDMN and IDN features from GLCM, RP from GLRLM, ZE from GLSZM, and DE from GLDM feature sets were the only features that were the most reproducible (COV < 5) against changes in all imaging settings. In addition, the IDMN feature from GLCM, LALGLE, SALGLE and LGLZE from GLSZM, and SDLGLE from GLDM feature sets were the features that were less reproducible (COV>20 ) against changes in all imaging settings. Matrix size has the greatest impact on feature variability as most of features are not repeatable and 82.76 of them had (COV>20 ).
Conclusion: Repeatability and reproducibility of SPECT radiomics texture features in different imaging settings is feature-dependent, and different image acquisitions and reconstructions have different effects on radiomics texture features. Low COV radiomics features could be consider for further clinical studies.
Keywords: SPECT, Radiomics, Cardiac, Repeatability, Reproducibility
△ Less
Submitted 11 September, 2019;
originally announced September 2019.
-
Non-Invasive MGMT Status Prediction in GBM Cancer Using Magnetic Resonance Images (MRI) Radiomics Features: Univariate and Multivariate Machine Learning Radiogenomics Analysis
Authors:
Ghasem Hajianfar,
Isaac Shiri,
Hassan Maleki,
Niki Oveisi,
Abbass Haghparast,
Hamid Abdollahi,
Mehrdad Oveisi
Abstract:
Background and aim: This study aimed to predict methylation status of the O-6 methyl guanine-DNA methyl transferase (MGMT) gene promoter status by using MRI radiomics features, as well as univariate and multivariate analysis.
Material and Methods: Eighty-two patients who had a MGMT methylation status were include in this study. Tumor were manually segmented in the four regions of MR images, a) w…
▽ More
Background and aim: This study aimed to predict methylation status of the O-6 methyl guanine-DNA methyl transferase (MGMT) gene promoter status by using MRI radiomics features, as well as univariate and multivariate analysis.
Material and Methods: Eighty-two patients who had a MGMT methylation status were include in this study. Tumor were manually segmented in the four regions of MR images, a) whole tumor, b) active/enhanced region, c) necrotic regions and d) edema regions (E). About seven thousand radiomics features were extracted for each patient. Feature selection and classifier were used to predict MGMT status through different machine learning algorithms. The area under the curve (AUC) of receiver operating characteristic (ROC) curve was used for model evaluations.
Results: Regarding univariate analysis, the Inverse Variance feature from gray level co-occurrence matrix (GLCM) in Whole Tumor segment with 4.5 mm Sigma of Laplacian of Gaussian filter with AUC: 0.71 (p-value: 0.002) was found to be the best predictor. For multivariate analysis, the decision tree classifier with Select from Model feature selector and LOG filter in Edema region had the highest performance (AUC: 0.78), followed by Ada Boost classifier with Select from Model feature selector and LOG filter in Edema region (AUC: 0.74).
Conclusion: This study showed that radiomics using machine learning algorithms is a feasible, noninvasive approach to predict MGMT methylation status in GBM cancer patients
Keywords: Radiomics, Radiogenomics, GBM, MRI, MGMT
△ Less
Submitted 8 July, 2019;
originally announced July 2019.
-
Next Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Approaches
Authors:
Isaac Shiri,
Hassan Maleki,
Ghasem Hajianfar,
Hamid Abdollahi,
Saeed Ashrafinia,
Mathieu Hatt,
Mehrdad Oveisi,
Arman Rahmim
Abstract:
Aim: In the present work, we aimed to evaluate a comprehensive radiomics framework that enabled prediction of EGFR and KRAS mutation status in NSCLC cancer patients based on PET and CT multi-modalities radiomic features and machine learning (ML) algorithms. Methods: Our study involved 211 NSCLC cancer patient with PET and CTD images. More than twenty thousand radiomic features from different image…
▽ More
Aim: In the present work, we aimed to evaluate a comprehensive radiomics framework that enabled prediction of EGFR and KRAS mutation status in NSCLC cancer patients based on PET and CT multi-modalities radiomic features and machine learning (ML) algorithms. Methods: Our study involved 211 NSCLC cancer patient with PET and CTD images. More than twenty thousand radiomic features from different image-feature sets were extracted Feature value was normalized to obtain Z-scores, followed by student t-test students for comparison, high correlated features were eliminated and the False discovery rate (FDR) correction were performed Six feature selection methods and twelve classifiers were used to predict gene status in patient and model evaluation was reported on independent validation sets (68 patients). Results: The best predictive power of conventional PET parameters was achieved by SUVpeak (AUC: 0.69, P-value = 0.0002) and MTV (AUC: 0.55, P-value = 0.0011) for EGFR and KRAS, respectively. Univariate analysis of radiomics features improved prediction power up to AUC: 75 (q-value: 0.003, Short Run Emphasis feature of GLRLM from LOG preprocessed image of PET with sigma value 1.5) and AUC: 0.71 (q-value 0.00005, The Large Dependence Low Gray Level Emphasis from GLDM in LOG preprocessed image of CTD sigma value 5) for EGFR and KRAS, respectively. Furthermore, the machine learning algorithm improved the perdition power up to AUC: 0.82 for EGFR (LOG preprocessed of PET image set with sigma 3 with VT feature selector and SGD classifier) and AUC: 0.83 for KRAS (CT image set with sigma 3.5 with SM feature selector and SGD classifier). Conclusion: We demonstrated that radiomic features extracted from different image-feature sets could be used for EGFR and KRAS mutation status prediction in NSCLC patients, and showed that they have more predictive power than conventional imaging parameters.
△ Less
Submitted 3 July, 2019;
originally announced July 2019.
-
A Novel Deep Learning Based Approach for Left Ventricle Segmentation in Echocardiography: MFP-Unet
Authors:
Shakiba Moradi,
Mostafa Ghelich-Oghli,
Azin Alizadehasl,
Isaac Shiri,
Niki Oveisi,
Mehrdad Oveisi,
Majid Maleki,
Jan Dhooge
Abstract:
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed t…
▽ More
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed this problem through an encoder-decoder path. Despite outstanding overall performance, U-net ignores the contribution of all semantic strengths in the segmentation procedure. In the present study, we have proposed a novel architecture to tackle this drawback. Feature maps in all levels of the decoder path of U-net are concatenated, their depths are equalized, and up-sampled to a fixed dimension. This stack of feature maps would be the input of the semantic segmentation layer. The proposed network yielded state-of-the-art results when comparing with results from U-net, dilated U-net, and deeplabv3, using the same dataset. An average Dice Metric (DM) of 0.945, Hausdorff Distance (HD) of 1.62, Jaccard Coefficient (JC) of 0.97, and Mean Absolute Distance (MAD) of 1.32 are achieved. The correlation graph, bland-altman analysis, and box plot showed a great agreement between automatic and manually calculated volume, area, and length.
△ Less
Submitted 22 December, 2019; v1 submitted 25 June, 2019;
originally announced June 2019.
-
Deep-Fill: Deep Learning Based Sinogram Domain Gap Filling in Positron Emission Tomography
Authors:
Isaac Shiri,
Peyman Sheikhzadeh,
Mohammad Reza Ay
Abstract:
One of the major challenges in design and develo** of PET, scanners are the presence of inactive areas between the detector blocks which degrade the image spatial resolution and leads to streaking artifacts especially when we employ analytical image reconstruction. The aim of this study is to assess the feasibility of generating the gap-free PET image using the deep convolutional encoder-decoder…
▽ More
One of the major challenges in design and develo** of PET, scanners are the presence of inactive areas between the detector blocks which degrade the image spatial resolution and leads to streaking artifacts especially when we employ analytical image reconstruction. The aim of this study is to assess the feasibility of generating the gap-free PET image using the deep convolutional encoder-decoder in sinogram space. The gap-corrupted sinograms of simulated HRRT scanner, sinograms without gaps as ideal/ground truth and predicted sinograms owing to the implemented our deep-fill method were quantitatively compared. In total, 1293 phantom images divided into three main sets of training 1000, 150 Validation, and 143 test set. The 1000 training images were augmented using affine transformations with various sub-transforms including rotation (rotate randomly), translation to 12000 Image with 6 frequencies of 2 main methods. The Deep-Fill architecture consists of an encoder and a decoder part, and it is composed of convolution operation, max pooling, ReLU activation, concatenation, and up convolution layers. The gap image is going through the network along with all possible paths then the gap-free image was generated by decoder part of the network. The quality of the generated images was quantitatively assessed by different quality metrics in both sinogram space and reconstruction images. We demonstrated that deep learning based approaches applied to inter-detector gap filling can recover the missing data in sinogram with high quantitative accuracy and have the potential to significantly improve the reconstructed image quality and prevent degradation of PET image quantification.
△ Less
Submitted 16 June, 2019;
originally announced June 2019.
-
PET/CT Radiomic Sequencer for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients
Authors:
Isaac Shiri,
Hassan Maleki,
Ghasem Hajianfar,
Hamid Abdollahi,
Saeed Ashrafinia,
Mathieu Hatt,
Mehrdad Oveisi,
Arman Rahmim
Abstract:
The aim of this study was to develop radiomic models using PET/CT radiomic features with different machine learning approaches for finding best predictive epidermal growth factor receptor (EGFR) and Kirsten rat sarcoma viral oncogene (KRAS) mutation status. Patients images including PET and CT [diagnostic (CTD) and low dose CT (CTA)] were pre-processed using wavelet (WAV), Laplacian of Gaussian (L…
▽ More
The aim of this study was to develop radiomic models using PET/CT radiomic features with different machine learning approaches for finding best predictive epidermal growth factor receptor (EGFR) and Kirsten rat sarcoma viral oncogene (KRAS) mutation status. Patients images including PET and CT [diagnostic (CTD) and low dose CT (CTA)] were pre-processed using wavelet (WAV), Laplacian of Gaussian (LOG) and 64 bin discretization (BIN) (alone or in combinations) and several features from images were extracted. The prediction performance of model was checked using the area under the receiver operator characteristic (ROC) curve (AUC). Results showed a wide range of radiomic model AUC performances up to 0.75 in prediction of EGFR and KRAS mutation status. Combination of K-Best and variance threshold feature selector with logistic regression (LREG) classifier in diagnostic CT scan led to the best performance in EGFR (CTD-BIN+B-KB+LREG, AUC: mean 0.75 sd 0.10) and KRAS (CTD-BIN-LOG-WAV+B-VT+LREG, AUC: mean 0.75 sd 0.07) respectively. Additionally, incorporating PET, kept AUC values at ~0.74. When considering conventional features only, highest predictive performance was achieved by PET SUVpeak (AUC: 0.69) for EGFR and by PET MTV (AUC: 0.55) for KRAS. In comparison with conventional PET parameters such as standard uptake value, radiomic models were found as more predictive. Our findings demonstrated that non-invasive and reliable radiomics analysis can be successfully used to predict EGFR and KRAS mutation status in NSCLC patients.
△ Less
Submitted 15 June, 2019;
originally announced June 2019.
