-
Graph Neural Networks in Histopathology: Emerging Trends and Future Directions
Authors:
Siemen Brussee,
Giorgio Buzzanca,
Anne M. R. Schrader,
Jesper Kers
Abstract:
Histopathological analysis of Whole Slide Images (WSIs) has seen a surge in the utilization of deep learning methods, particularly Convolutional Neural Networks (CNNs). However, CNNs often fall short in capturing the intricate spatial dependencies inherent in WSIs. Graph Neural Networks (GNNs) present a promising alternative, adept at directly modeling pairwise interactions and effectively discern…
▽ More
Histopathological analysis of Whole Slide Images (WSIs) has seen a surge in the utilization of deep learning methods, particularly Convolutional Neural Networks (CNNs). However, CNNs often fall short in capturing the intricate spatial dependencies inherent in WSIs. Graph Neural Networks (GNNs) present a promising alternative, adept at directly modeling pairwise interactions and effectively discerning the topological tissue and cellular structures within WSIs. Recognizing the pressing need for deep learning techniques that harness the topological structure of WSIs, the application of GNNs in histopathology has experienced rapid growth. In this comprehensive review, we survey GNNs in histopathology, discuss their applications, and explore emerging trends that pave the way for future advancements in the field. We begin by elucidating the fundamentals of GNNs and their potential applications in histopathology. Leveraging quantitative literature analysis, we identify four emerging trends: Hierarchical GNNs, Adaptive Graph Structure Learning, Multimodal GNNs, and Higher-order GNNs. Through an in-depth exploration of these trends, we offer insights into the evolving landscape of GNNs in histopathological analysis. Based on our findings, we propose future directions to propel the field forward. Our analysis serves to guide researchers and practitioners towards innovative approaches and methodologies, fostering advancements in histopathological analysis through the lens of graph neural networks.
△ Less
Submitted 21 June, 2024; v1 submitted 18 June, 2024;
originally announced June 2024.
-
Segmentation of diagnostic tissue compartments on whole slide images with renal thrombotic microangiopathies (TMAs)
Authors:
Huy Q. Vo,
Pietro A. Cicalese,
Surya Seshan,
Syed A. Rizvi,
Aneesh Vathul,
Gloria Bueno,
Anibal Pedraza Dorado,
Niels Grabe,
Katharina Stolle,
Francesco Pesce,
Joris J. T. H. Roelofs,
Jesper Kers,
Vitoantonio Bevilacqua,
Nicola Altini,
Bernd Schröppel,
Dario Roccatello,
Antonella Barreca,
Savino Sciascia,
Chandra Mohan,
Hien V. Nguyen,
Jan U. Becker
Abstract:
The thrombotic microangiopathies (TMAs) manifest in renal biopsy histology with a broad spectrum of acute and chronic findings. Precise diagnostic criteria for a renal biopsy diagnosis of TMA are missing. As a first step towards a machine learning- and computer vision-based analysis of wholes slide images from renal biopsies, we trained a segmentation model for the decisive diagnostic kidney tissu…
▽ More
The thrombotic microangiopathies (TMAs) manifest in renal biopsy histology with a broad spectrum of acute and chronic findings. Precise diagnostic criteria for a renal biopsy diagnosis of TMA are missing. As a first step towards a machine learning- and computer vision-based analysis of wholes slide images from renal biopsies, we trained a segmentation model for the decisive diagnostic kidney tissue compartments artery, arteriole, glomerulus on a set of whole slide images from renal biopsies with TMAs and Mimickers (distinct diseases with a similar nephropathological appearance as TMA like severe benign nephrosclerosis, various vasculitides, Bevacizumab-plug glomerulopathy, arteriolar light chain deposition disease). Our segmentation model combines a U-Net-based tissue detection with a Shifted windows-transformer architecture to reach excellent segmentation results for even the most severely altered glomeruli, arterioles and arteries, even on unseen staining domains from a different nephropathology lab. With accurate automatic segmentation of the decisive renal biopsy compartments in human renal vasculopathies, we have laid the foundation for large-scale compartment-specific machine learning and computer vision analysis of renal biopsy repositories with TMAs.
△ Less
Submitted 28 November, 2023; v1 submitted 25 November, 2023;
originally announced November 2023.
-
Advances in Kidney Biopsy Lesion Assessment through Dense Instance Segmentation
Authors:
Zhan Xiong,
Junling He,
Pieter Valkema,
Tri Q. Nguyen,
Maarten Naesens,
Jesper Kers,
Fons J. Verbeek
Abstract:
Renal biopsies are the gold standard for diagnosis of kidney diseases. Lesion scores made by renal pathologists are semi-quantitative and exhibit high inter-observer variability. Automating lesion classification within segmented anatomical structures can provide decision support in quantification analysis and reduce the inter-observer variability. Nevertheless, classifying lesions in regions-of-in…
▽ More
Renal biopsies are the gold standard for diagnosis of kidney diseases. Lesion scores made by renal pathologists are semi-quantitative and exhibit high inter-observer variability. Automating lesion classification within segmented anatomical structures can provide decision support in quantification analysis and reduce the inter-observer variability. Nevertheless, classifying lesions in regions-of-interest (ROIs) is clinically challenging due to (a) a large amount of densely packed anatomical objects (up to 1000), (b) class imbalance across different compartments (at least 3), (c) significant variation in object scales (i.e. sizes and shapes), and (d) the presence of multi-label lesions per anatomical structure. Existing models lack the capacity to address these complexities efficiently and generically. This paper presents \textbf{a generalized technical solution} for large-scale, multi-source datasets with diverse lesions. Our approach utilizes two sub-networks: dense instance segmentation and lesion classification. We introduce \textbf{DiffRegFormer}, an end-to-end dense instance segmentation model designed for multi-class, multi-scale objects within ROIs. Combining diffusion models, transformers, and RCNNs, DiffRegFormer efficiently recognizes over 500 objects across three anatomical classes (glomeruli, tubuli, arteries) within ROIs on a single NVIDIA GeForce RTX 3090 GPU. On a dataset of 303 ROIs (from 148 Jones' silver-stained renal WSIs), it outperforms state of art models, achieving AP of 52.1\% (detection) and 46.8\% (segmentation). Our lesion classification sub-network achieves 89.2\% precision and 64.6\% recall on 21889 object patches (from the 303 ROIs). Importantly, the model demonstrates direct domain transfer to PAS-stained WSIs without fine-tuning.
△ Less
Submitted 28 March, 2024; v1 submitted 29 September, 2023;
originally announced September 2023.
-
U-Net-and-a-half: Convolutional network for biomedical image segmentation using multiple expert-driven annotations
Authors:
Yichi Zhang,
Jesper Kers,
Clarissa A. Cassol,
Joris J. Roelofs,
Najia Idrees,
Alik Farber,
Samir Haroon,
Kevin P. Daly,
Suvranu Ganguli,
Vipul C. Chitalia,
Vijaya B. Kolachalama
Abstract:
Development of deep learning systems for biomedical segmentation often requires access to expert-driven, manually annotated datasets. If more than a single expert is involved in the annotation of the same images, then the inter-expert agreement is not necessarily perfect, and no single expert annotation can precisely capture the so-called ground truth of the regions of interest on all images. Also…
▽ More
Development of deep learning systems for biomedical segmentation often requires access to expert-driven, manually annotated datasets. If more than a single expert is involved in the annotation of the same images, then the inter-expert agreement is not necessarily perfect, and no single expert annotation can precisely capture the so-called ground truth of the regions of interest on all images. Also, it is not trivial to generate a reference estimate using annotations from multiple experts. Here we present a deep neural network, defined as U-Net-and-a-half, which can simultaneously learn from annotations performed by multiple experts on the same set of images. U-Net-and-a-half contains a convolutional encoder to generate features from the input images, multiple decoders that allow simultaneous learning from image masks obtained from annotations that were independently generated by multiple experts, and a shared low-dimensional feature space. To demonstrate the applicability of our framework, we used two distinct datasets from digital pathology and radiology, respectively. Specifically, we trained two separate models using pathologist-driven annotations of glomeruli on whole slide images of human kidney biopsies (10 patients), and radiologist-driven annotations of lumen cross-sections of human arteriovenous fistulae obtained from intravascular ultrasound images (10 patients), respectively. The models based on U-Net-and-a-half exceeded the performance of the traditional U-Net models trained on single expert annotations alone, thus expanding the scope of multitask learning in the context of biomedical image segmentation.
△ Less
Submitted 10 August, 2021;
originally announced August 2021.
