MITI Minimum Information guidelines for highly multiplexed tissue images
Authors:
Denis Schapiro,
Clarence Yapp,
Artem Sokolov,
Sheila M. Reynolds,
Yu-An Chen,
Damir Sudar,
Yubin Xie,
Jeremy L. Muhlich,
Raquel Arias-Camison,
Sarah Arena,
Adam J. Taylor,
Milen Nikolov,
Madison Tyler,
Jia-Ren Lin,
Erik A. Burlingame,
Human Tumor Atlas Network,
Young H. Chang,
Samouil L Farhi,
Vésteinn Thorsson,
Nithya Venkatamohan,
Julia L. Drewes,
Dana Pe'er,
David A. Gutman,
Markus D. Herrmann,
Nils Gehlenborg
, et al. (14 additional authors not shown)
Abstract:
The imminent release of tissue atlases combining multi-channel microscopy with single cell sequencing and other omics data from normal and diseased specimens creates an urgent need for data and metadata standards that guide data deposition, curation and release. We describe a Minimum Information about highly multiplexed Tissue Imaging (MITI) standard that applies best practices developed for genom…
▽ More
The imminent release of tissue atlases combining multi-channel microscopy with single cell sequencing and other omics data from normal and diseased specimens creates an urgent need for data and metadata standards that guide data deposition, curation and release. We describe a Minimum Information about highly multiplexed Tissue Imaging (MITI) standard that applies best practices developed for genomics and other microscopy data to highly multiplexed tissue images and traditional histology.
△ Less
Submitted 23 February, 2022; v1 submitted 21 August, 2021;
originally announced August 2021.
Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis
Authors:
Richard J. Chen,
Ming Y. Lu,
**gwen Wang,
Drew F. K. Williamson,
Scott J. Rodig,
Neal I. Lindeman,
Faisal Mahmood
Abstract:
Cancer diagnosis, prognosis, and therapeutic response predictions are based on morphological information from histology slides and molecular profiles from genomic data. However, most deep learning-based objective outcome prediction and grading paradigms are based on histology or genomics alone and do not make use of the complementary information in an intuitive manner. In this work, we propose Pat…
▽ More
Cancer diagnosis, prognosis, and therapeutic response predictions are based on morphological information from histology slides and molecular profiles from genomic data. However, most deep learning-based objective outcome prediction and grading paradigms are based on histology or genomics alone and do not make use of the complementary information in an intuitive manner. In this work, we propose Pathomic Fusion, an interpretable strategy for end-to-end multimodal fusion of histology image and genomic (mutations, CNV, RNA-Seq) features for survival outcome prediction. Our approach models pairwise feature interactions across modalities by taking the Kronecker product of unimodal feature representations and controls the expressiveness of each representation via a gating-based attention mechanism. Following supervised learning, we are able to interpret and saliently localize features across each modality, and understand how feature importance shifts when conditioning on multimodal input. We validate our approach using glioma and clear cell renal cell carcinoma datasets from the Cancer Genome Atlas (TCGA), which contains paired whole-slide image, genotype, and transcriptome data with ground truth survival and histologic grade labels. In a 15-fold cross-validation, our results demonstrate that the proposed multimodal fusion paradigm improves prognostic determinations from ground truth grading and molecular subty**, as well as unimodal deep networks trained on histology and genomic data alone. The proposed method establishes insight and theory on how to train deep networks on multimodal biomedical data in an intuitive manner, which will be useful for other problems in medicine that seek to combine heterogeneous data streams for understanding diseases and predicting response and resistance to treatment.
△ Less
Submitted 3 September, 2020; v1 submitted 18 December, 2019;
originally announced December 2019.
