RudolfV: A Foundation Model by Pathologists for Pathologists
Authors:
Jonas Dippel,
Barbara Feulner,
Tobias Winterhoff,
Timo Milbich,
Stephan Tietz,
Simon Schallenberg,
Gabriel Dernbach,
Andreas Kunft,
Simon Heinke,
Marie-Lisa Eich,
Julika Ribbat-Idel,
Rosemarie Krupar,
Philipp Anders,
Niklas Prenißl,
Philipp Jurmeister,
David Horst,
Lukas Ruff,
Klaus-Robert Müller,
Frederick Klauschen,
Maximilian Alber
Abstract:
Artificial intelligence has started to transform histopathology impacting clinical diagnostics and biomedical research. However, while many computational pathology approaches have been proposed, most current AI models are limited with respect to generalization, application variety, and handling rare diseases. Recent efforts introduced self-supervised foundation models to address these challenges,…
▽ More
Artificial intelligence has started to transform histopathology impacting clinical diagnostics and biomedical research. However, while many computational pathology approaches have been proposed, most current AI models are limited with respect to generalization, application variety, and handling rare diseases. Recent efforts introduced self-supervised foundation models to address these challenges, yet existing approaches do not leverage pathologist knowledge by design. In this study, we present a novel approach to designing foundation models for computational pathology, incorporating pathologist expertise, semi-automated data curation, and a diverse dataset from over 15 laboratories, including 58 tissue types, and encompassing 129 different histochemical and immunohistochemical staining modalities. We demonstrate that our model "RudolfV" surpasses existing state-of-the-art foundation models across different benchmarks focused on tumor microenvironment profiling, biomarker evaluation, and reference case search while exhibiting favorable robustness properties. Our study shows how domain-specific knowledge can increase the efficiency and performance of pathology foundation models and enable novel application areas.
△ Less
Submitted 11 June, 2024; v1 submitted 8 January, 2024;
originally announced January 2024.
Clustering disease trajectories in contrastive feature space for biomarker discovery in age-related macular degeneration
Authors:
Robbie Holland,
Oliver Leingang,
Christopher Holmes,
Philipp Anders,
Rebecca Kaye,
Sophie Riedl,
Johannes C. Paetzold,
Ivan Ezhov,
Hrvoje Bogunović,
Ursula Schmidt-Erfurth,
Lars Fritsche,
Hendrik P. N. Scholl,
Sobha Sivaprasad,
Andrew J. Lotery,
Daniel Rueckert,
Martin J. Menten
Abstract:
Age-related macular degeneration (AMD) is the leading cause of blindness in the elderly. Current grading systems based on imaging biomarkers only coarsely group disease stages into broad categories and are unable to predict future disease progression. It is widely believed that this is due to their focus on a single point in time, disregarding the dynamic nature of the disease. In this work, we pr…
▽ More
Age-related macular degeneration (AMD) is the leading cause of blindness in the elderly. Current grading systems based on imaging biomarkers only coarsely group disease stages into broad categories and are unable to predict future disease progression. It is widely believed that this is due to their focus on a single point in time, disregarding the dynamic nature of the disease. In this work, we present the first method to automatically discover biomarkers that capture temporal dynamics of disease progression. Our method represents patient time series as trajectories in a latent feature space built with contrastive learning. Then, individual trajectories are partitioned into atomic sub-sequences that encode transitions between disease states. These are clustered using a newly introduced distance metric. In quantitative experiments we found our method yields temporal biomarkers that are predictive of conversion to late AMD. Furthermore, these clusters were highly interpretable to ophthalmologists who confirmed that many of the clusters represent dynamics that have previously been linked to the progression of AMD, even though they are currently not included in any clinical grading system.
△ Less
Submitted 20 March, 2023; v1 submitted 11 January, 2023;
originally announced January 2023.
