-
Visual Grounding of Whole Radiology Reports for 3D CT Images
Authors:
Akimichi Ichinose,
Taro Hatsutani,
Keigo Nakamura,
Yoshiro Kitamura,
Satoshi Iizuka,
Edgar Simo-Serra,
Shoji Kido,
Noriyuki Tomiyama
Abstract:
Building a large-scale training dataset is an essential problem in the development of medical image recognition systems. Visual grounding techniques, which automatically associate objects in images with corresponding descriptions, can facilitate labeling of large number of images. However, visual grounding of radiology reports for CT images remains challenging, because so many kinds of anomalies a…
▽ More
Building a large-scale training dataset is an essential problem in the development of medical image recognition systems. Visual grounding techniques, which automatically associate objects in images with corresponding descriptions, can facilitate labeling of large number of images. However, visual grounding of radiology reports for CT images remains challenging, because so many kinds of anomalies are detectable via CT imaging, and resulting report descriptions are long and complex. In this paper, we present the first visual grounding framework designed for CT image and report pairs covering various body parts and diverse anomaly types. Our framework combines two components of 1) anatomical segmentation of images, and 2) report structuring. The anatomical segmentation provides multiple organ masks of given CT images, and helps the grounding model recognize detailed anatomies. The report structuring helps to accurately extract information regarding the presence, location, and type of each anomaly described in corresponding reports. Given the two additional image/report features, the grounding model can achieve better localization. In the verification process, we constructed a large-scale dataset with region-description correspondence annotations for 10,410 studies of 7,321 unique patients. We evaluated our framework using grounding accuracy, the percentage of correctly localized anomalies, as a metric and demonstrated that the combination of the anatomical segmentation and the report structuring improves the performance with a large margin over the baseline model (66.0% vs 77.8%). Comparison with the prior techniques also showed higher performance of our method.
△ Less
Submitted 7 December, 2023;
originally announced December 2023.
-
Quantitative evaluation of simultaneous spatial and temporal regularization in liver perfusion studies using low-dose dynamic contrast-enhanced CT
Authors:
Kenya Murase,
Atsushi Nakamoto,
Noriyuki Tomiyama
Abstract:
The purpose of this study was to quantitatively evaluate the performance of different simultaneous spatial and temporal regularizers in liver perfusion studies using low-dose dynamic contrast-enhanced computed tomography (DCE-CT). A digital liver phantom was used to simulate chronic liver disease (CLD) and hepatocellular carcinoma (HCC) based on clinical data. Low-dose DCE-CT images were reconstru…
▽ More
The purpose of this study was to quantitatively evaluate the performance of different simultaneous spatial and temporal regularizers in liver perfusion studies using low-dose dynamic contrast-enhanced computed tomography (DCE-CT). A digital liver phantom was used to simulate chronic liver disease (CLD) and hepatocellular carcinoma (HCC) based on clinical data. Low-dose DCE-CT images were reconstructed using regularizers and a primal-dual algorithm. Subsequently, hepatic perfusion parameter (HPP) images were generated using a dual-input single-compartment model and linear least-squares method. In the CLD model, the effect of regularizers on the input functions (IFs) was examined by calculating the areas under the curves (AUCs) of the IFs, and the HPP estimation accuracy was evaluated by calculating the error and coefficient of variation (CV) between the HPP values obtained by the above methods and true values. In the HCC model, the ratios of the mean HPP values inside and outside the tumor were calculated. The AUCs of IFs decreased with increasing regularization parameter (RP) values. Although the AUC of arterial IF did not significantly depend on the regularizers, that of portal IF did. The error and CV were reduced using low-rank and sparse decomposition (LRSD). Total generalized variation (TGV) combined with LRSD (LTGV) was generally superior to the other regularizers in terms of HPP estimation accuracy and range of available RP values in both the CLD and HCC models. However, striped artifacts were more remarkable in the HPP images obtained by the TGV and LTGV than in those obtained by the other regularizers. The results suggest that the LRSD and LTGV are useful for improving the accuracy of HPP estimation using low-dose DCE-CT and for enhancing its practicality. This study will help select a suitable regularizer and/or RP value for low-dose DCE-CT liver perfusion studies.
△ Less
Submitted 19 October, 2023;
originally announced October 2023.
-
Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographs
Authors:
Junya Sato,
Yuki Suzuki,
Tomohiro Wataya,
Daiki Nishigaki,
Kosuke Kita,
Kazuki Yamagata,
Noriyuki Tomiyama,
Shoji Kido
Abstract:
Large numbers of labeled medical images are essential for the accurate detection of anomalies, but manual annotation is labor-intensive and time-consuming. Self-supervised learning (SSL) is a training method to learn data-specific features without manual annotation. Several SSL-based models have been employed in medical image anomaly detection. These SSL methods effectively learn representations i…
▽ More
Large numbers of labeled medical images are essential for the accurate detection of anomalies, but manual annotation is labor-intensive and time-consuming. Self-supervised learning (SSL) is a training method to learn data-specific features without manual annotation. Several SSL-based models have been employed in medical image anomaly detection. These SSL methods effectively learn representations in several field-specific images, such as natural and industrial product images. However, owing to the requirement of medical expertise, typical SSL-based models are inefficient in medical image anomaly detection. We present an SSL-based model that enables anatomical structure-based unsupervised anomaly detection (UAD). The model employs the anatomy-aware pasting (AnatPaste) augmentation tool. AnatPaste employs a threshold-based lung segmentation pretext task to create anomalies in normal chest radiographs, which are used for model pretraining. These anomalies are similar to real anomalies and help the model recognize them. We evaluate our model on three opensource chest radiograph datasets. Our model exhibit area under curves (AUC) of 92.1%, 78.7%, and 81.9%, which are the highest among existing UAD models. This is the first SSL model to employ anatomical information as a pretext task. AnatPaste can be applied in various deep learning models and downstream tasks. It can be employed for other modalities by fixing appropriate segmentation. Our code is publicly available at: https://github.com/jun-sato/AnatPaste.
△ Less
Submitted 11 June, 2022; v1 submitted 9 May, 2022;
originally announced May 2022.
-
Weak Supervision in Convolutional Neural Network for Semantic Segmentation of Diffuse Lung Diseases Using Partially Annotated Dataset
Authors:
Yuki Suzuki,
Kazuki Yamagata,
Yanagawa Masahiro,
Shoji Kido,
Noriyuki Tomiyama
Abstract:
Computer-aided diagnosis system for diffuse lung diseases (DLDs) is necessary for the objective assessment of the lung diseases. In this paper, we develop semantic segmentation model for 5 kinds of DLDs. DLDs considered in this work are consolidation, ground glass opacity, honeycombing, emphysema, and normal. Convolutional neural network (CNN) is one of the most promising technique for semantic se…
▽ More
Computer-aided diagnosis system for diffuse lung diseases (DLDs) is necessary for the objective assessment of the lung diseases. In this paper, we develop semantic segmentation model for 5 kinds of DLDs. DLDs considered in this work are consolidation, ground glass opacity, honeycombing, emphysema, and normal. Convolutional neural network (CNN) is one of the most promising technique for semantic segmentation among machine learning algorithms. While creating annotated dataset for semantic segmentation is laborious and time consuming, creating partially annotated dataset, in which only one chosen class is annotated for each image, is easier since annotators only need to focus on one class at a time during the annotation task. In this paper, we propose a new weak supervision technique that effectively utilizes partially annotated dataset. The experiments using partially annotated dataset composed 372 CT images demonstrated that our proposed technique significantly improved segmentation accuracy.
△ Less
Submitted 26 March, 2020; v1 submitted 27 February, 2020;
originally announced February 2020.
