-
An efficient deep neural network to find small objects in large 3D images
Authors:
Jungkyu Park,
Jakub Chłędowski,
Stanisław Jastrzębski,
Jan Witowski,
Yanqi Xu,
Linda Du,
Sushma Gaddam,
Eric Kim,
Alana Lewin,
Ujas Parikh,
Anastasia Plaunova,
Sardius Chen,
Alexandra Millet,
James Park,
Kristine Pysarenko,
Shalin Patel,
Julia Goldberg,
Melanie Wegener,
Linda Moy,
Laura Heacock,
Beatriu Reig,
Krzysztof J. Geras
Abstract:
3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alt…
▽ More
3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alternative, a neural network that enables efficient classification of full-resolution 3D medical images. Compared to off-the-shelf convolutional neural networks, our network, 3D Globally-Aware Multiple Instance Classifier (3D-GMIC), uses 77.98%-90.05% less GPU memory and 91.23%-96.02% less computation. While it is trained only with image-level labels, without segmentation labels, it explains its predictions by providing pixel-level saliency maps. On a dataset collected at NYU Langone Health, including 85,526 patients with full-field 2D mammography (FFDM), synthetic 2D mammography, and 3D mammography, 3D-GMIC achieves an AUC of 0.831 (95% CI: 0.769-0.887) in classifying breasts with malignant findings using 3D mammography. This is comparable to the performance of GMIC on FFDM (0.816, 95% CI: 0.737-0.878) and synthetic 2D (0.826, 95% CI: 0.754-0.884), which demonstrates that 3D-GMIC successfully classified large 3D images despite focusing computation on a smaller percentage of its input compared to GMIC. Therefore, 3D-GMIC identifies and utilizes extremely small regions of interest from 3D images consisting of hundreds of millions of pixels, dramatically reducing associated computational challenges. 3D-GMIC generalizes well to BCS-DBT, an external dataset from Duke University Hospital, achieving an AUC of 0.848 (95% CI: 0.798-0.896).
△ Less
Submitted 26 February, 2023; v1 submitted 16 October, 2022;
originally announced October 2022.
-
Differences between human and machine perception in medical diagnosis
Authors:
Taro Makino,
Stanislaw Jastrzebski,
Witold Oleszkiewicz,
Celin Chacko,
Robin Ehrenpreis,
Naziya Samreen,
Chloe Chhor,
Eric Kim,
Jiyon Lee,
Kristine Pysarenko,
Beatriu Reig,
Hildegard Toth,
Divya Awal,
Linda Du,
Alice Kim,
James Park,
Daniel K. Sodickson,
Laura Heacock,
Linda Moy,
Kyunghyun Cho,
Krzysztof J. Geras
Abstract:
Deep neural networks (DNNs) show promise in image-based medical diagnosis, but cannot be fully trusted since their performance can be severely degraded by dataset shifts to which human perception remains invariant. If we can better understand the differences between human and machine perception, we can potentially characterize and mitigate this effect. We therefore propose a framework for comparin…
▽ More
Deep neural networks (DNNs) show promise in image-based medical diagnosis, but cannot be fully trusted since their performance can be severely degraded by dataset shifts to which human perception remains invariant. If we can better understand the differences between human and machine perception, we can potentially characterize and mitigate this effect. We therefore propose a framework for comparing human and machine perception in medical diagnosis. The two are compared with respect to their sensitivity to the removal of clinically meaningful information, and to the regions of an image deemed most suspicious. Drawing inspiration from the natural image domain, we frame both comparisons in terms of perturbation robustness. The novelty of our framework is that separate analyses are performed for subgroups with clinically meaningful differences. We argue that this is necessary in order to avert Simpson's paradox and draw correct conclusions. We demonstrate our framework with a case study in breast cancer screening, and reveal significant differences between radiologists and DNNs. We compare the two with respect to their robustness to Gaussian low-pass filtering, performing a subgroup analysis on microcalcifications and soft tissue lesions. For microcalcifications, DNNs use a separate set of high frequency components than radiologists, some of which lie outside the image regions considered most suspicious by radiologists. These features run the risk of being spurious, but if not, could represent potential new biomarkers. For soft tissue lesions, the divergence between radiologists and DNNs is even starker, with DNNs relying heavily on spurious high frequency components ignored by radiologists. Importantly, this deviation in soft tissue lesions was only observable through subgroup analysis, which highlights the importance of incorporating medical domain knowledge into our comparison framework.
△ Less
Submitted 27 November, 2020;
originally announced November 2020.
-
Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening
Authors:
Nan Wu,
Jason Phang,
Jungkyu Park,
Yiqiu Shen,
Zhe Huang,
Masha Zorin,
Stanisław Jastrzębski,
Thibault Févry,
Joe Katsnelson,
Eric Kim,
Stacey Wolfson,
Ujas Parikh,
Sushma Gaddam,
Leng Leng Young Lin,
Kara Ho,
Joshua D. Weinstein,
Beatriu Reig,
Yiming Gao,
Hildegard Toth,
Kristine Pysarenko,
Alana Lewin,
Jiyon Lee,
Krystal Airola,
Eralda Mema,
Stephanie Chung
, et al. (7 additional authors not shown)
Abstract:
We present a deep convolutional neural network for breast cancer screening exam classification, trained and evaluated on over 200,000 exams (over 1,000,000 images). Our network achieves an AUC of 0.895 in predicting whether there is a cancer in the breast, when tested on the screening population. We attribute the high accuracy of our model to a two-stage training procedure, which allows us to use…
▽ More
We present a deep convolutional neural network for breast cancer screening exam classification, trained and evaluated on over 200,000 exams (over 1,000,000 images). Our network achieves an AUC of 0.895 in predicting whether there is a cancer in the breast, when tested on the screening population. We attribute the high accuracy of our model to a two-stage training procedure, which allows us to use a very high-capacity patch-level network to learn from pixel-level labels alongside a network learning from macroscopic breast-level labels. To validate our model, we conducted a reader study with 14 readers, each reading 720 screening mammogram exams, and find our model to be as accurate as experienced radiologists when presented with the same data. Finally, we show that a hybrid model, averaging probability of malignancy predicted by a radiologist with a prediction of our neural network, is more accurate than either of the two separately. To better understand our results, we conduct a thorough analysis of our network's performance on different subpopulations of the screening population, model design, training procedure, errors, and properties of its internal representations.
△ Less
Submitted 19 March, 2019;
originally announced March 2019.
