-
Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment
Authors:
Zhiyuan Li,
Hailong Li,
Anca L. Ralescu,
Jonathan R. Dillman,
Mekibib Altaye,
Kim M. Cecil,
Nehal A. Parikh,
Lili He
Abstract:
The integration of different imaging modalities, such as structural, diffusion tensor, and functional magnetic resonance imaging, with deep learning models has yielded promising outcomes in discerning phenotypic characteristics and enhancing disease diagnosis. The development of such a technique hinges on the efficient fusion of heterogeneous multimodal features, which initially reside within dist…
▽ More
The integration of different imaging modalities, such as structural, diffusion tensor, and functional magnetic resonance imaging, with deep learning models has yielded promising outcomes in discerning phenotypic characteristics and enhancing disease diagnosis. The development of such a technique hinges on the efficient fusion of heterogeneous multimodal features, which initially reside within distinct representation spaces. Naively fusing the multimodal features does not adequately capture the complementary information and could even produce redundancy. In this work, we present a novel joint self-supervised and supervised contrastive learning method to learn the robust latent feature representation from multimodal MRI data, allowing the projection of heterogeneous features into a shared common space, and thereby amalgamating both complementary and analogous information across various modalities and among similar subjects. We performed a comparative analysis between our proposed method and alternative deep multimodal learning approaches. Through extensive experiments on two independent datasets, the results demonstrated that our method is significantly superior to several other deep multimodal learning methods in predicting abnormal neurodevelopment. Our method has the capability to facilitate computer-aided diagnosis within clinical practice, harnessing the power of multimodal data.
△ Less
Submitted 22 December, 2023;
originally announced December 2023.
-
Semantic Segmentation of Porosity in 4D Spatio-Temporal X-ray μCT of Titanium Coated Ni wires using Deep Learning
Authors:
Pradyumna Elavarthi,
Arun Bhattacharjee,
Ashley Paz y Puente,
Anca Ralescu
Abstract:
A fully convolutional neural network was used to measure the evolution of the volume fraction of two different Kirkendall pores during the homogenization of Ti coated Ni wires. Traditional methods like Otsus thresholding and the largest connected component analysis were used to obtain the masks for training the segmentation model. Once trained, the model was used to semantically segment the two ty…
▽ More
A fully convolutional neural network was used to measure the evolution of the volume fraction of two different Kirkendall pores during the homogenization of Ti coated Ni wires. Traditional methods like Otsus thresholding and the largest connected component analysis were used to obtain the masks for training the segmentation model. Once trained, the model was used to semantically segment the two types of pores at different stages in their evolution. Masks of the pores predicted by the network were then used to measure the volume fraction of porosity at 0 mins, 240 mins, and 480 mins of homogenization. The model predicted an increase in porosity for one type of pore and a decrease in porosity for another type of pore due to pore sintering, and it achieved an F1 Score of 0.95.
△ Less
Submitted 24 June, 2023;
originally announced June 2023.
-
A Novel Collaborative Self-Supervised Learning Method for Radiomic Data
Authors:
Zhiyuan Li,
Hailong Li,
Anca L. Ralescu,
Jonathan R. Dillman,
Nehal A. Parikh,
Lili He
Abstract:
The computer-aided disease diagnosis from radiomic data is important in many medical applications. However, develo** such a technique relies on annotating radiological images, which is a time-consuming, labor-intensive, and expensive process. In this work, we present the first novel collaborative self-supervised learning method to solve the challenge of insufficient labeled radiomic data, whose…
▽ More
The computer-aided disease diagnosis from radiomic data is important in many medical applications. However, develo** such a technique relies on annotating radiological images, which is a time-consuming, labor-intensive, and expensive process. In this work, we present the first novel collaborative self-supervised learning method to solve the challenge of insufficient labeled radiomic data, whose characteristics are different from text and image data. To achieve this, we present two collaborative pretext tasks that explore the latent pathological or biological relationships between regions of interest and the similarity and dissimilarity information between subjects. Our method collaboratively learns the robust latent feature representations from radiomic data in a self-supervised manner to reduce human annotation efforts, which benefits the disease diagnosis. We compared our proposed method with other state-of-the-art self-supervised learning methods on a simulation study and two independent datasets. Extensive experimental results demonstrated that our method outperforms other self-supervised learning methods on both classification and regression tasks. With further refinement, our method shows the potential advantage in automatic disease diagnosis with large-scale unlabeled data available.
△ Less
Submitted 20 February, 2023;
originally announced February 2023.
