Showing 1–7 of 7 results for author: Kelkar, V A

Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics

Authors: Rucha Deshpande, Varun A. Kelkar, Dimitrios Gotsis, Prabhat Kc, Rong** Zeng, Kyle J. Myers, Frank J. Brooks, Mark A. Anastasio

Abstract: The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. The goal of this challenge was to promote the development of deep generative models (DGMs) for medical imaging and to emphasize the need for their domain-relevant assessment via the analysis of relevant image statistics. As part of this Grand Challeng… ▽ More The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. The goal of this challenge was to promote the development of deep generative models (DGMs) for medical imaging and to emphasize the need for their domain-relevant assessment via the analysis of relevant image statistics. As part of this Grand Challenge, a training dataset was developed based on 3D anthropomorphic breast phantoms from the VICTRE virtual imaging toolbox. A two-stage evaluation procedure consisting of a preliminary check for memorization and image quality (based on the Frechet Inception distance (FID)), and a second stage evaluating the reproducibility of image statistics corresponding to domain-relevant radiomic features was developed. A summary measure was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, and to identify various artifacts. 58 submissions from 12 unique users were received for this Challenge. The top-ranked submission employed a conditional latent diffusion model, whereas the joint runners-up employed a generative adversarial network, followed by another network for image superresolution. We observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID-based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. This Grand Challenge highlighted the need for domain-specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Iso-entropy partially coherent optical fields that cannot be inter-converted unitarily

Authors: Mitchell Harling, Varun A. Kelkar, Kimani C. Toussaint, Jr., Ayman F. Abouraddy

Abstract: For partially coherent optical fields in which a single binary degree of freedom (DoF) is relevant, such as polarization, entropy uniquely identifies the class of optical fields that can be converted into each other via unitary transformations. However, when multiple DoFs are taken into consideration, entropy no longer serves this purpose. We investigate the structure of the family of iso-entropy… ▽ More For partially coherent optical fields in which a single binary degree of freedom (DoF) is relevant, such as polarization, entropy uniquely identifies the class of optical fields that can be converted into each other via unitary transformations. However, when multiple DoFs are taken into consideration, entropy no longer serves this purpose. We investigate the structure of the family of iso-entropy partially coherent optical fields defined by two binary DoFs (polarization and two spatial modes) and described by a 4x4 coherence matrix G. We find that the rank of G (the number of its non-zero eigenvalues) plays a critical role in this context: whereby any pair of iso-entropy rank-2 fields can be converted into each other unitarily, this is not necessarily the case for a pair of rank-3 or rank-4 fields. Furthermore, unitary transformations between iso-entropy fields of different ranks are strictly forbidden. Instead, such conversions require entropy-maintaining non-unitary transformations that potentially combine filtering projections and randomizing operations. We experimentally synthesize partially coherent iso-entropy optical fields of all ranks, and tomographically reconstruct their coherence matrices. Moreover, we steer the coherence matrix over iso-entropy trajectories that maintain a fixed rank (intra-rank conversion) or that involve changes in the rank (inter-rank conversion). These findings offer a new perspective for the potential utility of partially coherent light in optical communications and sensing. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Locked entropy in partially coherent fields

Authors: Mitchell Harling, Varun A. Kelkar, Kimani C. Toussaint, Jr., Ayman F. Abouraddy

Abstract: We introduce a taxonomy for partially coherent optical fields spanning multiple degrees of freedom (DoFs) based on the rank of the associated coherence matrix (the number of non-zero eigenvalues). When DoFs comprise two spatial modes and polarization, a fourfold classification emerges, with rank-1 fields corresponding to fully coherent fields. We demonstrate theoretically and confirm experimentall… ▽ More We introduce a taxonomy for partially coherent optical fields spanning multiple degrees of freedom (DoFs) based on the rank of the associated coherence matrix (the number of non-zero eigenvalues). When DoFs comprise two spatial modes and polarization, a fourfold classification emerges, with rank-1 fields corresponding to fully coherent fields. We demonstrate theoretically and confirm experimentally that these classes have heretofore unrecognized different properties. Specifically, whereas rank-2 fields can always be rendered separable with respect to its DoFs via a unitary transformation, rank-3 fields are always non-separable. Consequently, the entropy for a rank-2 field can always be concentrated into a single DoF (thus ridding the other DoF of statistical fluctuations), whereas some entropy is always 'locked' in one DoF of a rank-3 field. △ Less

Submitted 21 October, 2023; originally announced October 2023.

Assessing the ability of generative adversarial networks to learn canonical medical image statistics

Authors: Varun A. Kelkar, Dimitrios S. Gotsis, Frank J. Brooks, Prabhat KC, Kyle J. Myers, Rong** Zeng, Mark A. Anastasio

Abstract: In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn t… ▽ More In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) that are relevant to objective assessment of image quality is investigated. It is shown that although the employed GAN successfully learned several basic first- and second-order statistics of the specific medical SIMs under consideration and generated images with high perceptual quality, it failed to correctly learn several per-image statistics pertinent to the these SIMs, highlighting the urgent need to assess medical image GANs in terms of objective measures of image quality. △ Less

Submitted 26 April, 2022; v1 submitted 25 April, 2022; originally announced April 2022.

Evaluating Procedures for Establishing Generative Adversarial Network-based Stochastic Image Models in Medical Imaging

Authors: Varun A. Kelkar, Dimitrios S. Gotsis, Frank J. Brooks, Kyle J. Myers, Prabhat KC, Rong** Zeng, Mark A. Anastasio

Abstract: Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several areas of medical imaging, such as unconditional medical image synthesis, image restoration, reconstruction and translation, and optimization of imaging systems. However, procedures for establishing stochastic image models (SIMs) using GANs remain generic and do not address specific issues… ▽ More Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several areas of medical imaging, such as unconditional medical image synthesis, image restoration, reconstruction and translation, and optimization of imaging systems. However, procedures for establishing stochastic image models (SIMs) using GANs remain generic and do not address specific issues relevant to medical imaging. In this work, canonical SIMs that simulate realistic vessels in angiography images are employed to evaluate procedures for establishing SIMs using GANs. The GAN-based SIM is compared to the canonical SIM based on its ability to reproduce those statistics that are meaningful to the particular medically realistic SIM considered. It is shown that evaluating GANs using classical metrics and medically relevant metrics may lead to different conclusions about the fidelity of the trained GANs. This work highlights the need for the development of objective metrics for evaluating GANs. △ Less

Submitted 7 April, 2022; originally announced April 2022.

Comments: Published in SPIE Medical Imaging 2022: Image Perception, Observer Performance, and Technology Assessment

Prior image-based medical image reconstruction using a style-based generative adversarial network

Authors: Varun A. Kelkar, Mark A. Anastasio

Abstract: Computed medical imaging systems require a computational reconstruction procedure for image formation. In order to recover a useful estimate of the object to-be-imaged when the recorded measurements are incomplete, prior knowledge about the nature of object must be utilized. In order to improve the conditioning of an ill-posed imaging inverse problem, deep learning approaches are being actively in… ▽ More Computed medical imaging systems require a computational reconstruction procedure for image formation. In order to recover a useful estimate of the object to-be-imaged when the recorded measurements are incomplete, prior knowledge about the nature of object must be utilized. In order to improve the conditioning of an ill-posed imaging inverse problem, deep learning approaches are being actively investigated for better representing object priors and constraints. This work proposes to use a style-based generative adversarial network (StyleGAN) to constrain an image reconstruction problem in the case where additional information in the form of a prior image of the sought-after object is available. An optimization problem is formulated in the intermediate latent-space of a StyleGAN, that is disentangled with respect to meaningful image attributes or "styles", such as the contrast used in magnetic resonance imaging (MRI). Discrepancy between the sought-after and prior images is measured in the disentangled latent-space, and is used to regularize the inverse problem in the form of constraints on specific styles of the disentangled latent-space. A stylized numerical study inspired by MR imaging is designed, where the sought-after and the prior image are structurally similar, but belong to different contrast mechanisms. The presented numerical studies demonstrate the superiority of the proposed approach as compared to classical approaches in the form of traditional metrics. △ Less

Submitted 17 February, 2022; originally announced February 2022.

Impact of deep learning-based image super-resolution on binary signal detection

Authors: Xiaohui Zhang, Varun A. Kelkar, Jason Granstedt, Hua Li, Mark A. Anastasio

Abstract: Deep learning-based image super-resolution (DL-SR) has shown great promise in medical imaging applications. To date, most of the proposed methods for DL-SR have only been assessed by use of traditional measures of image quality (IQ) that are commonly employed in the field of computer vision. However, the impact of these methods on objective measures of image quality that are relevant to medical im… ▽ More Deep learning-based image super-resolution (DL-SR) has shown great promise in medical imaging applications. To date, most of the proposed methods for DL-SR have only been assessed by use of traditional measures of image quality (IQ) that are commonly employed in the field of computer vision. However, the impact of these methods on objective measures of image quality that are relevant to medical imaging tasks remains largely unexplored. In this study, we investigate the impact of DL-SR methods on binary signal detection performance. Two popular DL-SR methods, the super-resolution convolutional neural network (SRCNN) and the super-resolution generative adversarial network (SRGAN), were trained by use of simulated medical image data. Binary signal-known-exactly with background-known-statistically (SKE/BKS) and signal-known-statistically with background-known-statistically (SKS/BKS) detection tasks were formulated. Numerical observers, which included a neural network-approximated ideal observer and common linear numerical observers, were employed to assess the impact of DL-SR on task performance. The impact of the complexity of the DL-SR network architectures on task-performance was quantified. In addition, the utility of DL-SR for improving the task-performance of sub-optimal observers was investigated. Our numerical experiments confirmed that, as expected, DL-SR could improve traditional measures of IQ. However, for many of the study designs considered, the DL-SR methods provided little or no improvement in task performance and could even degrade it. It was observed that DL-SR could improve the task-performance of sub-optimal observers under certain conditions. The presented study highlights the urgent need for the objective assessment of DL-SR methods and suggests avenues for improving their efficacy in medical imaging applications. △ Less

Submitted 5 July, 2021; originally announced July 2021.