-
Common and Rare Fundus Diseases Identification Using Vision-Language Foundation Model with Knowledge of Over 400 Diseases
Authors:
Meng Wang,
Tian Lin,
Aidi Lin,
Kai Yu,
Yuanyuan Peng,
Lianyu Wang,
Cheng Chen,
Ke Zou,
Huiyu Liang,
Man Chen,
Xue Yao,
Meiqin Zhang,
Binwei Huang,
Chaoxin Zheng,
Peixin Zhang,
Wei Chen,
Yilong Luo,
Yifan Chen,
Honghe Xia,
Tingkun Shi,
Qi Zhang,
**ming Guo,
Xiaolin Chen,
**gcheng Wang,
Yih Chung Tham
, et al. (24 additional authors not shown)
Abstract:
Previous foundation models for retinal images were pre-trained with limited disease categories and knowledge base. Here we introduce RetiZero, a vision-language foundation model that leverages knowledge from over 400 fundus diseases. To RetiZero's pre-training, we compiled 341,896 fundus images paired with text descriptions, sourced from public datasets, ophthalmic literature, and online resources…
▽ More
Previous foundation models for retinal images were pre-trained with limited disease categories and knowledge base. Here we introduce RetiZero, a vision-language foundation model that leverages knowledge from over 400 fundus diseases. To RetiZero's pre-training, we compiled 341,896 fundus images paired with text descriptions, sourced from public datasets, ophthalmic literature, and online resources, encompassing a diverse range of diseases across multiple ethnicities and countries. RetiZero exhibits superior performance in several downstream tasks, including zero-shot disease recognition, image-to-image retrieval, and internal- and cross-domain disease identification. In zero-shot scenarios, RetiZero achieves Top5 accuracy scores of 0.8430 for 15 fundus diseases and 0.7561 for 52 fundus diseases. For image retrieval, it achieves Top5 scores of 0.9500 and 0.8860 for the same disease sets, respectively. Clinical evaluations show that RetiZero's Top3 zero-shot performance surpasses the average of 19 ophthalmologists from Singapore, China and the United States. Furthermore, RetiZero significantly enhances clinicians' accuracy in diagnosing fundus disease. These findings underscore the value of integrating the RetiZero foundation model into clinical settings, where a variety of fundus diseases are encountered.
△ Less
Submitted 30 June, 2024; v1 submitted 13 June, 2024;
originally announced June 2024.
-
Unpaired Optical Coherence Tomography Angiography Image Super-Resolution via Frequency-Aware Inverse-Consistency GAN
Authors:
Weiwen Zhang,
Dawei Yang,
Haoxuan Che,
An Ran Ran,
Carol Y. Cheung,
Hao Chen
Abstract:
For optical coherence tomography angiography (OCTA) images, a limited scanning rate leads to a trade-off between field-of-view (FOV) and imaging resolution. Although larger FOV images may reveal more parafoveal vascular lesions, their application is greatly hampered due to lower resolution. To increase the resolution, previous works only achieved satisfactory performance by using paired data for t…
▽ More
For optical coherence tomography angiography (OCTA) images, a limited scanning rate leads to a trade-off between field-of-view (FOV) and imaging resolution. Although larger FOV images may reveal more parafoveal vascular lesions, their application is greatly hampered due to lower resolution. To increase the resolution, previous works only achieved satisfactory performance by using paired data for training, but real-world applications are limited by the challenge of collecting large-scale paired images. Thus, an unpaired approach is highly demanded. Generative Adversarial Network (GAN) has been commonly used in the unpaired setting, but it may struggle to accurately preserve fine-grained capillary details, which are critical biomarkers for OCTA. In this paper, our approach aspires to preserve these details by leveraging the frequency information, which represents details as high-frequencies ($\textbf{hf}$) and coarse-grained backgrounds as low-frequencies ($\textbf{lf}$). In general, we propose a GAN-based unpaired super-resolution method for OCTA images and exceptionally emphasize $\textbf{hf}$ fine capillaries through a dual-path generator. To facilitate a precise spectrum of the reconstructed image, we also propose a frequency-aware adversarial loss for the discriminator and introduce a frequency-aware focal consistency loss for end-to-end optimization. Experiments show that our method outperforms other state-of-the-art unpaired methods both quantitatively and visually.
△ Less
Submitted 29 September, 2023;
originally announced September 2023.
-
Reference-based OCT Angiogram Super-resolution with Learnable Texture Generation
Authors:
Yuyan Ruan,
Dawei Yang,
Ziqi Tang,
An Ran Ran,
Carol Y. Cheung,
Hao Chen
Abstract:
Optical coherence tomography angiography (OCTA) is a new imaging modality to visualize retinal microvasculature and has been readily adopted in clinics. High-resolution OCT angiograms are important to qualitatively and quantitatively identify potential biomarkers for different retinal diseases accurately. However, one significant problem of OCTA is the inevitable decrease in resolution when increa…
▽ More
Optical coherence tomography angiography (OCTA) is a new imaging modality to visualize retinal microvasculature and has been readily adopted in clinics. High-resolution OCT angiograms are important to qualitatively and quantitatively identify potential biomarkers for different retinal diseases accurately. However, one significant problem of OCTA is the inevitable decrease in resolution when increasing the field-of-view given a fixed acquisition time. To address this issue, we propose a novel reference-based super-resolution (RefSR) framework to preserve the resolution of the OCT angiograms while increasing the scanning area. Specifically, textures from the normal RefSR pipeline are used to train a learnable texture generator (LTG), which is designed to generate textures according to the input. The key difference between the proposed method and traditional RefSR models is that the textures used during inference are generated by the LTG instead of being searched from a single reference image. Since the LTG is optimized throughout the whole training process, the available texture space is significantly enlarged and no longer limited to a single reference image, but extends to all textures contained in the training samples. Moreover, our proposed LTGNet does not require a reference image at the inference phase, thereby becoming invulnerable to the selection of the reference image. Both experimental and visual results show that LTGNet has superior performance and robustness over state-of-the-art methods, indicating good reliability and promise in real-life deployment. The source code will be made available upon acceptance.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
Colors of an Earth-like exoplanet -- Temporal flux and polarization signals of the Earth
Authors:
A. Groot,
L. Rossi,
V. J. H. Trees,
J. C. Y. Cheung,
D. M. Stam
Abstract:
Understanding the total flux and polarization signals of Earth-like planets and their spectral and temporal variability is essential for the future characterization of such exoplanets. We provide computed total (F) and linearly (Q and U) and circularly (V) polarized fluxes, and the degree of polarization P of sunlight that is reflected by a model Earth, to be used for instrument designs, optimizin…
▽ More
Understanding the total flux and polarization signals of Earth-like planets and their spectral and temporal variability is essential for the future characterization of such exoplanets. We provide computed total (F) and linearly (Q and U) and circularly (V) polarized fluxes, and the degree of polarization P of sunlight that is reflected by a model Earth, to be used for instrument designs, optimizing observational strategies, and/or develo** retrieval algorithms. We modeled a realistic Earth-like planet using one year of daily Earth-observation data: cloud parameters (distribution, optical thickness, top pressure, and particle effective radius), and surface parameters (distribution, surface type, and albedo). The Stokes vector of the disk-averaged reflected sunlight was computed for phase angles alpha from 0 to 180 degrees, and for wavelengths lambda from 350 to 865 nm. The total flux F is one order of magnitude higher than the polarized flux Q, and Q is two and four orders of magnitude higher than U and V, respectively. Without clouds, the peak-to-peak daily variations due to the planetary rotation increase with increasing lambda for F, Q, and P, while they decrease for U and V. Clouds modify but do not completely suppress the variations that are due to rotating surface features. With clouds, the variation in F increases with increasing lambda, while in Q, it decreases with increasing lambda, except at the largest phase angles. In earlier work, it was shown that with oceans, Q changes color from blue through white to red. The alpha where the color changes increases with increasing cloud coverage. Here, we show that this unique color change in Q also occurs when the oceans are partly replaced by continents, with or without clouds. The degree of polarization P shows a similar color change. Our computed fluxes and degree of polarization will be made publicly available.
△ Less
Submitted 30 July, 2020;
originally announced July 2020.
-
Finding New Diagnostic Information for Detecting Glaucoma using Neural Networks
Authors:
Erfan Noury,
Suria S. Mannil,
Robert T. Chang,
An Ran Ran,
Carol Y. Cheung,
Suman S. Thapa,
Harsha L. Rao,
Srilakshmi Dasari,
Mohammed Riyazuddin,
Dolly Chang,
Sriharsha Nagaraj,
Clement C. Tham,
Reza Zadeh
Abstract:
We describe a new approach to automated Glaucoma detection in 3D Spectral Domain Optical Coherence Tomography (OCT) optic nerve scans. First, we gathered a unique and diverse multi-ethnic dataset of OCT scans consisting of glaucoma and non-glaucomatous cases obtained from four tertiary care eye hospitals located in four different countries. Using this longitudinal data, we achieved state-of-the-ar…
▽ More
We describe a new approach to automated Glaucoma detection in 3D Spectral Domain Optical Coherence Tomography (OCT) optic nerve scans. First, we gathered a unique and diverse multi-ethnic dataset of OCT scans consisting of glaucoma and non-glaucomatous cases obtained from four tertiary care eye hospitals located in four different countries. Using this longitudinal data, we achieved state-of-the-art results for automatically detecting Glaucoma from a single raw OCT using a 3D Deep Learning system. These results are close to human doctors in a variety of settings across heterogeneous datasets and scanning environments. To verify correctness and interpretability of the automated categorization, we used saliency maps to find areas of focus for the model. Matching human doctor behavior, the model predictions indeed correlated with the conventional diagnostic parameters in the OCT printouts, such as the retinal nerve fiber layer. We further used our model to find new areas in the 3D data that are presently not being identified as a diagnostic parameter to detect glaucoma by human doctors. Namely, we found that the Lamina Cribrosa (LC) region can be a valuable source of helpful diagnostic information previously unavailable to doctors during routine clinical care because it lacks a quantitative printout. Our model provides such volumetric quantification of this region. We found that even when a majority of the RNFL is removed, the LC region can distinguish glaucoma. This is clinically relevant in high myopes, when the RNFL is already reduced, and thus the LC region may help differentiate glaucoma in this confounding situation. We further generalize this approach to create a new algorithm called DiagFind that provides a recipe for finding new diagnostic information in medical imagery that may have been previously unusable by doctors.
△ Less
Submitted 2 September, 2020; v1 submitted 14 October, 2019;
originally announced October 2019.
-
Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening
Authors:
Xi Wang,
Hao Chen,
Luyang Luo,
An-ran Ran,
Poemen P. Chan,
Clement C. Tham,
Carol Y. Cheung,
Pheng-Ann Heng
Abstract:
Optical Coherence Tomography (OCT) imaging plays an important role in glaucoma diagnosis in clinical practice. Early detection and timely treatment can prevent glaucoma patients from permanent vision loss. However, only a dearth of automated methods has been developed based on OCT images for glaucoma study. In this paper, we present a novel framework to effectively classify glaucoma OCT images fro…
▽ More
Optical Coherence Tomography (OCT) imaging plays an important role in glaucoma diagnosis in clinical practice. Early detection and timely treatment can prevent glaucoma patients from permanent vision loss. However, only a dearth of automated methods has been developed based on OCT images for glaucoma study. In this paper, we present a novel framework to effectively classify glaucoma OCT images from normal ones. A semi-supervised learning strategy with smoothness assumption is applied for surrogate assignment of missing function regression labels. Besides, the proposed multi-task learning network is capable of exploring the structure and function relationship from the OCT image and visual field measurement simultaneously, which contributes to classification performance boosting. Essentially, we are the first to unify the structure analysis and function regression for glaucoma screening. It is also worth noting that we build the largest glaucoma OCT image dataset involving 4877 volumes to develop and evaluate the proposed method. Extensive experiments demonstrate that our framework outperforms the baseline methods and two glaucoma experts by a large margin, achieving 93.2%, 93.2% and 97.8% on accuracy, F1 score and AUC, respectively.
△ Less
Submitted 25 July, 2019;
originally announced July 2019.
-
Simultaneous Detection of Multiple Change Points and Community Structures in Time Series of Networks
Authors:
Rex C. Y. Cheung,
Alexander Aue,
Seungyong Hwang,
Thomas C. M. Lee
Abstract:
In many complex systems, networks and graphs arise in a natural manner. Often, time evolving behavior can be easily found and modeled using time-series methodology. Amongst others, two common research problems in network analysis are community detection and change-point detection. Community detection aims at finding specific sub-structures within the networks, and change-point detection tries to f…
▽ More
In many complex systems, networks and graphs arise in a natural manner. Often, time evolving behavior can be easily found and modeled using time-series methodology. Amongst others, two common research problems in network analysis are community detection and change-point detection. Community detection aims at finding specific sub-structures within the networks, and change-point detection tries to find the time points at which sub-structures change. We propose a novel methodology to detect both community structures and change points simultaneously based on a model selection framework in which the Minimum Description Length Principle (MDL) is utilized as minimizing objective criterion. The promising practical performance of the proposed method is illustrated via a series of numerical experiments and real data analysis.
△ Less
Submitted 30 June, 2020; v1 submitted 29 November, 2018;
originally announced December 2018.
-
Piecewise quantile autoregressive modeling for nonstationary time series
Authors:
Alexander Aue,
Rex C. Y. Cheung,
Thomas C. M. Lee,
Ming Zhong
Abstract:
We develop a new methodology for the fitting of nonstationary time series that exhibit nonlinearity, asymmetry, local persistence and changes in location scale and shape of the underlying distribution. In order to achieve this goal, we perform model selection in the class of piecewise stationary quantile autoregressive processes. The best model is defined in terms of minimizing a minimum descripti…
▽ More
We develop a new methodology for the fitting of nonstationary time series that exhibit nonlinearity, asymmetry, local persistence and changes in location scale and shape of the underlying distribution. In order to achieve this goal, we perform model selection in the class of piecewise stationary quantile autoregressive processes. The best model is defined in terms of minimizing a minimum description length criterion derived from an asymmetric Laplace likelihood. Its practical minimization is done with the use of genetic algorithms. If the data generating process follows indeed a piecewise quantile autoregression structure, we show that our method is consistent for estimating the break points and the autoregressive parameters. Empirical work suggests that the proposed method performs well in finite samples.
△ Less
Submitted 28 September, 2016;
originally announced September 2016.
-
Consistent Estimation for Partition-wise Regression and Classification Models
Authors:
Rex C. Y. Cheung,
Alexander Aue,
Thomas C. M. Lee
Abstract:
Partition-wise models offer a flexible approach for modeling complex and multidimensional data that are capable of producing interpretable results. They are based on partitioning the observed data into regions, each of which is modeled with a simple submodel. The success of this approach highly depends on the quality of the partition, as too large a region could lead to a non-simple submodel, whil…
▽ More
Partition-wise models offer a flexible approach for modeling complex and multidimensional data that are capable of producing interpretable results. They are based on partitioning the observed data into regions, each of which is modeled with a simple submodel. The success of this approach highly depends on the quality of the partition, as too large a region could lead to a non-simple submodel, while too small a region could inflate estimation variance. This paper proposes an automatic procedure for choosing the partition (i.e., the number of regions and the boundaries between regions) as well as the submodels for the regions. It is shown that, under the assumption of the existence of a true partition, the proposed partition estimator is statistically consistent. The methodology is demonstrated for both regression and classification problems.
△ Less
Submitted 11 January, 2016;
originally announced January 2016.
-
$B -> πl ν$ Form Factors Calculated on the Light-Front
Authors:
C. Y. Cheung,
C. W. Hwang,
W. M. Zhang
Abstract:
A consistent treatment of $B\rightarrow πl ν$ decay is given on the light-front. The $B$ to $π$ transition form factors are calculated in the entire physical range of momentum transfer for the first time. The valence-quark contribution is obtained using relativistic light-front wave functions. Higher quark-antiquark Fock-state of the $B$-meson bound state is represented effectively by the…
▽ More
A consistent treatment of $B\rightarrow πl ν$ decay is given on the light-front. The $B$ to $π$ transition form factors are calculated in the entire physical range of momentum transfer for the first time. The valence-quark contribution is obtained using relativistic light-front wave functions. Higher quark-antiquark Fock-state of the $B$-meson bound state is represented effectively by the $|B^*π\rangle$ configuration, and its effect is calculated in the chiral perturbation theory. Wave function renormalization is taken into account consistently. The $|B^*π\rangle$ contribution dominates near the zero-recoil point ($q^2\simeq 25$ GeV$^2$), and decreases rapidly as the recoil momentum increases. We find that the calculated form factor $f_+(q^2)$ follows approximately a dipole $q^2$-dependence in the entire range of momentum transfer.
△ Less
Submitted 24 April, 1996; v1 submitted 14 February, 1996;
originally announced February 1996.
-
Corrections to Chiral Dynamics of Heavy Hadrons: (I) 1/M Correction
Authors:
H. Y. Cheng,
C. Y. Cheung,
G. L. Lin,
Y. C. Lin,
T. M. Yan,
H. L. Yu
Abstract:
In earlier publications we have analyzed the strong and radiative decays of heavy hadrons in a formalism which incorporates both heavy-quark and chiral symmetries. In particular, we have derived a heavy-hadron chiral Lagrangian whose coupling constants are related by the heavy-quark flavor-spin symmetry arising from the QCD Lagrangian with infinitely massive quarks. In this paper, we re-examine…
▽ More
In earlier publications we have analyzed the strong and radiative decays of heavy hadrons in a formalism which incorporates both heavy-quark and chiral symmetries. In particular, we have derived a heavy-hadron chiral Lagrangian whose coupling constants are related by the heavy-quark flavor-spin symmetry arising from the QCD Lagrangian with infinitely massive quarks. In this paper, we re-examine the structure of the above chiral Lagrangian by including the effects of $1/m_Q$ corrections in the heavy quark effective theory. The relations among the coupling constants, originally derived in the heavy-quark limit, are modified by heavy quark symmetry breaking interactions in QCD. Some of the implications are discussed.
△ Less
Submitted 17 August, 1993;
originally announced August 1993.
-
Chiral Lagrangians for Radiative Decays of Heavy Hadrons
Authors:
H. Y. Cheng,
C. Y. Cheung,
G. L. Lin,
Y. C. Lin,
T. M. Yan,
H. L. Yu
Abstract:
The radiative decays of heavy mesons and heavy baryons are studied in a formalism which incorporates both the heavy quark symmetry and the chiral symmetry. The chiral Lagrangians for the electromagnetic interactions of heavy hadrons consist of two pieces: one from gauging electromagnetically the strong-interaction chiral Lagrangian, and the other from the anomalous magnetic moment interactions o…
▽ More
The radiative decays of heavy mesons and heavy baryons are studied in a formalism which incorporates both the heavy quark symmetry and the chiral symmetry. The chiral Lagrangians for the electromagnetic interactions of heavy hadrons consist of two pieces: one from gauging electromagnetically the strong-interaction chiral Lagrangian, and the other from the anomalous magnetic moment interactions of the heavy baryons and mesons. Due to the heavy quark spin symmetry, the latter contains only one independent coupling constant in the meson sector and two in the baryon sector. These coupling constants only depend on the light quarks and can be calculated in the nonrelativistic quark model. However, the charm quark is not heavy enough and the contribution from its magnetic moment must be included. Applications to the radiative decays $D^\ast \rightarrow D γ~,~B^\ast \rightarrow B γ~,~ Ξ^\prime_c \rightarrow Ξ_c γ~, Σ_c \rightarrow Λ_c γ$ and $Σ_c \rightarrow Λ_c πγ$ are given. Together with our previous results on the strong decay rates of $D^\ast \rightarrow D π$ and $Σ_c \rightarrow Λ_c π$, predictions are obtained for the total widths and branching ratios of $D^\ast$ and $Σ_c$. The decays $Σ^+_c \rightarrow Λ^+_c π^0 γ$ and $Σ^0_c \rightarrow Λ^+_c π^- γ$ are discussed to illustrate the important roles played by both the heavy quark symmetry and the chiral symmetry.
△ Less
Submitted 21 September, 1992;
originally announced September 1992.
