A foundation model utilizing chest CT volumes and radiology reports for supervised-level zero-shot detection of abnormalities
Authors:
Ibrahim Ethem Hamamci,
Sezgin Er,
Furkan Almas,
Ayse Gulnihan Simsek,
Sevval Nil Esirgun,
Irem Dogan,
Muhammed Furkan Dasdelen,
Bastian Wittmann,
Enis Simsar,
Mehmet Simsar,
Emine Bensu Erdemir,
Abdullah Alanbay,
Anjany Sekuboyina,
Berkan Lafci,
Mehmet K. Ozdemir,
Bjoern Menze
Abstract:
A major challenge in computational research in 3D medical imaging is the lack of comprehensive datasets. Addressing this issue, our study introduces CT-RATE, the first 3D medical imaging dataset that pairs images with textual reports. CT-RATE consists of 25,692 non-contrast chest CT volumes, expanded to 50,188 through various reconstructions, from 21,304 unique patients, along with corresponding r…
▽ More
A major challenge in computational research in 3D medical imaging is the lack of comprehensive datasets. Addressing this issue, our study introduces CT-RATE, the first 3D medical imaging dataset that pairs images with textual reports. CT-RATE consists of 25,692 non-contrast chest CT volumes, expanded to 50,188 through various reconstructions, from 21,304 unique patients, along with corresponding radiology text reports. Leveraging CT-RATE, we developed CT-CLIP, a CT-focused contrastive language-image pre-training framework. As a versatile, self-supervised model, CT-CLIP is designed for broad application and does not require task-specific training. Remarkably, CT-CLIP outperforms state-of-the-art, fully supervised methods in multi-abnormality detection across all key metrics, thus eliminating the need for manual annotation. We also demonstrate its utility in case retrieval, whether using imagery or textual queries, thereby advancing knowledge dissemination. The open-source release of CT-RATE and CT-CLIP marks a significant advancement in medical AI, enhancing 3D imaging analysis and fostering innovation in healthcare.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
