-
Deepfake histological images for enhancing digital pathology
Authors:
Kianoush Falahkheirkhah,
Saumya Tiwari,
Kevin Yeh,
Sounak Gupta,
Loren Herrera-Hernandez,
Michael R. McCarthy,
Rafael E. Jimenez,
John C. Cheville,
Rohit Bhargava
Abstract:
An optical microscopic examination of thinly cut stained tissue on glass slides prepared from a FFPE tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of any pathologist is dependent on their direct experience with common as well as rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high…
▽ More
An optical microscopic examination of thinly cut stained tissue on glass slides prepared from a FFPE tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of any pathologist is dependent on their direct experience with common as well as rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task and artificially synthesized histological images can prove greatly beneficial. Here, we present an approach to not only generate histological images that reproduce the diagnostic morphologic features of common disease but also provide a user ability to generate new and rare morphologies. Our approach involves develo** a generative adversarial network model that synthesizes pathology images constrained by class labels. We investigated the ability of this framework in synthesizing realistic prostate and colon tissue images and assessed the utility of these images in augmenting diagnostic ability of machine learning methods as well as their usability by a panel of experienced anatomic pathologists. Synthetic data generated by our framework performed similar to real data in training a deep learning model for diagnosis. Pathologists were not able to distinguish between real and synthetic images and showed a similar level of inter-observer agreement for prostate cancer grading. We extended the approach to significantly more complex images from colon biopsies and showed that the complex microenvironment in such tissues can also be reproduced. Finally, we present the ability for a user to generate deepfake histological images via a simple markup of sematic labels.
△ Less
Submitted 16 June, 2022;
originally announced June 2022.
-
A generative adversarial approach to facilitate archival-quality histopathologic diagnoses from frozen tissue sections
Authors:
Kianoush Falahkheirkhah,
Tao Guo,
Michael Hwang,
Pheroze Tamboli,
Christopher G Wood,
Jose A Karam,
Kanishka Sircar,
Rohit Bhargava
Abstract:
In clinical diagnostics and research involving histopathology, formalin fixed paraffin embedded (FFPE) tissue is almost universally favored for its superb image quality. However, tissue processing time (more than 24 hours) can slow decision-making. In contrast, fresh frozen (FF) processing (less than 1 hour) can yield rapid information but diagnostic accuracy is suboptimal due to lack of clearing,…
▽ More
In clinical diagnostics and research involving histopathology, formalin fixed paraffin embedded (FFPE) tissue is almost universally favored for its superb image quality. However, tissue processing time (more than 24 hours) can slow decision-making. In contrast, fresh frozen (FF) processing (less than 1 hour) can yield rapid information but diagnostic accuracy is suboptimal due to lack of clearing, morphologic deformation and more frequent artifacts. Here, we bridge this gap using artificial intelligence. We synthesize FFPE-like images ,virtual FFPE, from FF images using a generative adversarial network (GAN) from 98 paired kidney samples derived from 40 patients. Five board-certified pathologists evaluated the results in a blinded test. Image quality of the virtual FFPE data was assessed to be high and showed a close resemblance to real FFPE images. Clinical assessments of disease on the virtual FFPE images showed a higher inter-observer agreement compared to FF images. The nearly instantaneously generated virtual FFPE images can not only reduce time to information but can facilitate more precise diagnosis from routine FF images without extraneous costs and effort.
△ Less
Submitted 24 August, 2021;
originally announced August 2021.
-
Composition-Aware Spectroscopic Tomography
Authors:
Luke Pfister,
Rohit Bhargava,
Yoram Bresler,
P. Scott Carney
Abstract:
Chemical imaging provides information about the distribution of chemicals within a target. When combined with structural information about the target, in situ chemical imaging opens the door to applications ranging from tissue classification to industrial process monitoring. The combination of infrared spectroscopy and optical microscopy is a powerful tool for chemical imaging of thin targets. Unf…
▽ More
Chemical imaging provides information about the distribution of chemicals within a target. When combined with structural information about the target, in situ chemical imaging opens the door to applications ranging from tissue classification to industrial process monitoring. The combination of infrared spectroscopy and optical microscopy is a powerful tool for chemical imaging of thin targets. Unfortunately, extending this technique to targets with appreciable depth is prohibitively slow.
We combine confocal microscopy and infrared spectroscopy to provide chemical imaging in three spatial dimensions. Interferometric measurements are acquired at a small number of focal depths, and images are formed by solving a regularized inverse scattering problem. A low-dimensional signal model is key to our approach: we assume the target comprises a finite number of distinct chemical species. We establish conditions on the constituent spectra and the number of measurements needed for unique recovery of the target. Simulations illustrate imaging of cellular phantoms and sub-wavelength targets from noisy measurements.
△ Less
Submitted 4 September, 2020; v1 submitted 27 December, 2019;
originally announced December 2019.
-
A deep learning framework for morphologic detail beyond the diffraction limit in infrared spectroscopic imaging
Authors:
Kianoush Falahkheirkhah,
Kevin Yeh,
Shachi Mittal,
Luke Pfister,
Rohit Bhargava
Abstract:
Infrared (IR) microscopes measure spectral information that quantifies molecular content to assign the identity of biomedical cells but lack the spatial quality of optical microscopy to appreciate morphologic features. Here, we propose a method to utilize the semantic information of cellular identity from IR imaging with the morphologic detail of pathology images in a deep learning-based approach…
▽ More
Infrared (IR) microscopes measure spectral information that quantifies molecular content to assign the identity of biomedical cells but lack the spatial quality of optical microscopy to appreciate morphologic features. Here, we propose a method to utilize the semantic information of cellular identity from IR imaging with the morphologic detail of pathology images in a deep learning-based approach to image super-resolution. Using Generative Adversarial Networks (GANs), we enhance the spatial detail in IR imaging beyond the diffraction limit while retaining their spectral contrast. This technique can be rapidly integrated with modern IR microscopes to provide a framework useful for routine pathology.
△ Less
Submitted 19 December, 2019; v1 submitted 6 November, 2019;
originally announced November 2019.
