-
PLUTO: Pathology-Universal Transformer
Authors:
Dinkar Juyal,
Harshith Padigela,
Chintan Shah,
Daniel Shenker,
Natalia Harguindeguy,
Yi Liu,
Blake Martin,
Yibo Zhang,
Michael Nercessian,
Miles Markey,
Isaac Finberg,
Kelsey Luu,
Daniel Borders,
Syed Ashar Javed,
Emma Krause,
Raymond Biju,
Aashish Sood,
Allen Ma,
Jackson Nyman,
John Shamshoian,
Guillaume Chhor,
Darpan Sanghavi,
Marc Thibault,
Limin Yu,
Fedaa Najdawi
, et al. (8 additional authors not shown)
Abstract:
Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this wor…
▽ More
Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Synthetic DOmain-Targeted Augmentation (S-DOTA) Improves Model Generalization in Digital Pathology
Authors:
Sai Chowdary Gullapally,
Yibo Zhang,
Nitin Kumar Mittal,
Deeksha Kartik,
Sandhya Srinivasan,
Kevin Rose,
Daniel Shenker,
Dinkar Juyal,
Harshith Padigela,
Raymond Biju,
Victor Minden,
Chirag Maheshwari,
Marc Thibault,
Zvi Goldstein,
Luke Novak,
Nidhi Chandra,
Justin Lee,
Aaditya Prakash,
Chintan Shah,
John Abel,
Darren Fahy,
Amaro Taylor-Weiner,
Anand Sampat
Abstract:
Machine learning algorithms have the potential to improve patient outcomes in digital pathology. However, generalization of these tools is currently limited by sensitivity to variations in tissue preparation, staining procedures and scanning equipment that lead to domain shift in digitized slides. To overcome this limitation and improve model generalization, we studied the effectiveness of two Syn…
▽ More
Machine learning algorithms have the potential to improve patient outcomes in digital pathology. However, generalization of these tools is currently limited by sensitivity to variations in tissue preparation, staining procedures and scanning equipment that lead to domain shift in digitized slides. To overcome this limitation and improve model generalization, we studied the effectiveness of two Synthetic DOmain-Targeted Augmentation (S-DOTA) methods, namely CycleGAN-enabled Scanner Transform (ST) and targeted Stain Vector Augmentation (SVA), and compared them against the International Color Consortium (ICC) profile-based color calibration (ICC Cal) method and a baseline method using traditional brightness, color and noise augmentations. We evaluated the ability of these techniques to improve model generalization to various tasks and settings: four models, two model types (tissue segmentation and cell classification), two loss functions, six labs, six scanners, and three indications (hepatocellular carcinoma (HCC), nonalcoholic steatohepatitis (NASH), prostate adenocarcinoma). We compared these methods based on the macro-averaged F1 scores on in-distribution (ID) and out-of-distribution (OOD) test sets across multiple domains, and found that S-DOTA methods (i.e., ST and SVA) led to significant improvements over ICC Cal and baseline on OOD data while maintaining comparable performance on ID data. Thus, we demonstrate that S-DOTA may help address generalization due to domain shift in real world applications.
△ Less
Submitted 3 May, 2023;
originally announced May 2023.
-
Comparison of pharmacist evaluation of medication orders with predictions of a machine learning model
Authors:
Sophie-Camille Hogue,
Flora Chen,
Geneviève Brassard,
Denis Lebel,
Jean-François Bussières,
Audrey Durand,
Maxime Thibault
Abstract:
The objective of this work was to assess the clinical performance of an unsupervised machine learning model aimed at identifying unusual medication orders and pharmacological profiles. We conducted a prospective study between April 2020 and August 2020 where 25 clinical pharmacists dichotomously (typical or atypical) rated 12,471 medication orders and 1,356 pharmacological profiles. Based on AUPR,…
▽ More
The objective of this work was to assess the clinical performance of an unsupervised machine learning model aimed at identifying unusual medication orders and pharmacological profiles. We conducted a prospective study between April 2020 and August 2020 where 25 clinical pharmacists dichotomously (typical or atypical) rated 12,471 medication orders and 1,356 pharmacological profiles. Based on AUPR, performance was poor for orders, but satisfactory for profiles. Pharmacists considered the model a useful screening tool.
△ Less
Submitted 3 November, 2020;
originally announced November 2020.
-
ColorUNet: A convolutional classification approach to colorization
Authors:
Vincent Billaut,
Matthieu de Rochemonteix,
Marc Thibault
Abstract:
This paper tackles the challenge of colorizing grayscale images. We take a deep convolutional neural network approach, and choose to take the angle of classification, working on a finite set of possible colors. Similarly to a recent paper, we implement a loss and a prediction function that favor realistic, colorful images rather than "true" ones.
We show that a rather lightweight architecture in…
▽ More
This paper tackles the challenge of colorizing grayscale images. We take a deep convolutional neural network approach, and choose to take the angle of classification, working on a finite set of possible colors. Similarly to a recent paper, we implement a loss and a prediction function that favor realistic, colorful images rather than "true" ones.
We show that a rather lightweight architecture inspired by the U-Net, and trained on a reasonable amount of pictures of landscapes, achieves satisfactory results on this specific subset of pictures. We show that data augmentation significantly improves the performance and robustness of the model, and provide visual analysis of the prediction confidence.
We show an application of our model, extending the task to video colorization. We suggest a way to smooth color predictions across frames, without the need to train a recurrent network designed for sequential inputs.
△ Less
Submitted 7 November, 2018;
originally announced November 2018.
-
Time and spectrum-resolving mutiphoton correlator for 300-900 nm
Authors:
Kelsey D. Johnsen,
Piotr Kolenderski,
Carmelo Scarcella,
Marilyne Thibault,
Alberto Tosi,
Thomas Jennewein
Abstract:
We demonstrate a single-photon sensitive spectrometer in the visible range, which allows us to perform time-resolved and multi-photon spectral correlation measurements. It is based on a monochromator composed of two gratings, collimation optics and an array of single photon avalanche diodes. The time resolution can reach $110$ ps and the spectral resolution is $2$ nm/pixel, limited by the design o…
▽ More
We demonstrate a single-photon sensitive spectrometer in the visible range, which allows us to perform time-resolved and multi-photon spectral correlation measurements. It is based on a monochromator composed of two gratings, collimation optics and an array of single photon avalanche diodes. The time resolution can reach $110$ ps and the spectral resolution is $2$ nm/pixel, limited by the design of the monochromator. This technique can easily be combined with commercial monochromators, and can be useful for joint spectrum measurements of two photons emitted in the process of parametric down conversion, as well as time-resolved spectrum measurements in optical coherence tomography or medical physics applications.
△ Less
Submitted 17 June, 2014;
originally announced June 2014.
-
On the geometric nature of characteristic classes of surface bundles
Authors:
Thomas Church,
Benson Farb,
Matthew Thibault
Abstract:
Each Morita--Mumford--Miller (MMM) class e_n assigns to each genus g >= 2 surface bundle S_g -> E^{2n+2} -> M^{2n} an integer e_n^#(E -> M) := <e_n,[M]> in Z. We prove that when n is odd the number e_n^#(E -> M) depends only on the diffeomorphism type of E, not on g, M, or the map E -> M. More generally, we prove that e_n^#(E -> M) depends only on the cobordism class of E. Recent work of Hatcher i…
▽ More
Each Morita--Mumford--Miller (MMM) class e_n assigns to each genus g >= 2 surface bundle S_g -> E^{2n+2} -> M^{2n} an integer e_n^#(E -> M) := <e_n,[M]> in Z. We prove that when n is odd the number e_n^#(E -> M) depends only on the diffeomorphism type of E, not on g, M, or the map E -> M. More generally, we prove that e_n^#(E -> M) depends only on the cobordism class of E. Recent work of Hatcher implies that this stronger statement is false when n is even. If E -> M is a holomorphic fibering of complex manifolds, we show that for every n the number e_n^#(E -> M) only depends on the complex cobordism type of E.
We give a general procedure to construct manifolds fibering as surface bundles in multiple ways, providing infinitely many examples to which our theorems apply. As an application of our results we give a new proof of the rational case of a recent theorem of Giansiracusa--Tillmann that the odd MMM classes e_{2i-1} vanish for any surface bundle which bounds a handlebody bundle. We show how the MMM classes can be seen as obstructions to low-genus fiberings. Finally, we discuss a number of open questions that arise from this work.
△ Less
Submitted 3 April, 2012; v1 submitted 1 March, 2011;
originally announced March 2011.
-
Toward a classification of finite quandles
Authors:
G. Ehrman,
A. Gurpinar,
M. Thibault,
D. N. Yetter
Abstract:
This paper summarizes substantive new results derived by a student team (the first three authors) under the direction of the fourth author at the 2005 session of the KSU REU ``Brainstorming and Barnstorming''. The main results are a decomposition theorem for quandles in terms of an operation of `semidisjoint union' showing that all finite quandles canonically decompose via iterated semidisjoint…
▽ More
This paper summarizes substantive new results derived by a student team (the first three authors) under the direction of the fourth author at the 2005 session of the KSU REU ``Brainstorming and Barnstorming''. The main results are a decomposition theorem for quandles in terms of an operation of `semidisjoint union' showing that all finite quandles canonically decompose via iterated semidisjoint unions into connected subquandles, and a structure theorem for finite connected quandles with prescribe inner automorphism group. The latter theorem suggests a new approach to the classification of finite connected quandles.
△ Less
Submitted 6 December, 2005;
originally announced December 2005.
-
On the Finite-Dimensional Irreducible Representations of PSL2(Z)
Authors:
Melinda G. Moran,
Matthew J. Thibault
Abstract:
We classify up to equivalence all finite-dimensional irreducible representations of PSL2(Z) whose restriction to the commutator subgroup is diagonalizable.
We classify up to equivalence all finite-dimensional irreducible representations of PSL2(Z) whose restriction to the commutator subgroup is diagonalizable.
△ Less
Submitted 19 July, 2005;
originally announced July 2005.