Showing 1–42 of 42 results for author: Williamson, K

HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis

Authors: Guillaume Jaume, Paul Doucet, Andrew H. Song, Ming Y. Lu, Cristina Almagro-Pérez, Sophia J. Wagner, Anurag J. Vaidya, Richard J. Chen, Drew F. K. Williamson, Ahrong Kim, Faisal Mahmood

Abstract: Spatial transcriptomics (ST) enables interrogating the molecular composition of tissue with ever-increasing resolution, depth, and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology as reflected by H&E-stained whole slide images (WSIs) encodes r… ▽ More Spatial transcriptomics (ST) enables interrogating the molecular composition of tissue with ever-increasing resolution, depth, and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology as reflected by H&E-stained whole slide images (WSIs) encodes rich information often overlooked in ST studies. Here, we introduce HEST-1k, a collection of 1,108 spatial transcriptomic profiles, each linked to a WSI and metadata. HEST-1k was assembled using HEST-Library from 131 public and internal cohorts encompassing 25 organs, two species (Homo Sapiens and Mus Musculus), and 320 cancer samples from 25 cancer types. HEST-1k processing enabled the identification of 1.5 million expression--morphology pairs and 60 million nuclei. HEST-1k is tested on three use cases: (1) benchmarking foundation models for histopathology (HEST-Benchmark), (2) biomarker identification, and (3) multimodal representation learning. HEST-1k, HEST-Library, and HEST-Benchmark can be freely accessed via https://github.com/mahmoodlab/hest. △ Less

Submitted 23 June, 2024; originally announced June 2024.

Comments: Under review

Morphological Prototy** for Unsupervised Slide Representation Learning in Computational Pathology

Authors: Andrew H. Song, Richard J. Chen, Tong Ding, Drew F. K. Williamson, Guillaume Jaume, Faisal Mahmood

Abstract: Representation learning of pathology whole-slide images (WSIs) has been has primarily relied on weak supervision with Multiple Instance Learning (MIL). However, the slide representations resulting from this approach are highly tailored to specific clinical tasks, which limits their expressivity and generalization, particularly in scenarios with limited data. Instead, we hypothesize that morphologi… ▽ More Representation learning of pathology whole-slide images (WSIs) has been has primarily relied on weak supervision with Multiple Instance Learning (MIL). However, the slide representations resulting from this approach are highly tailored to specific clinical tasks, which limits their expressivity and generalization, particularly in scenarios with limited data. Instead, we hypothesize that morphological redundancy in tissue can be leveraged to build a task-agnostic slide representation in an unsupervised fashion. To this end, we introduce PANTHER, a prototype-based approach rooted in the Gaussian mixture model that summarizes the set of WSI patches into a much smaller set of morphological prototypes. Specifically, each patch is assumed to have been generated from a mixture distribution, where each mixture component represents a morphological exemplar. Utilizing the estimated mixture parameters, we then construct a compact slide representation that can be readily used for a wide range of downstream tasks. By performing an extensive evaluation of PANTHER on subty** and survival tasks using 13 datasets, we show that 1) PANTHER outperforms or is on par with supervised MIL baselines and 2) the analysis of morphological prototypes brings new qualitative and quantitative insights into model interpretability. △ Less

Submitted 19 May, 2024; originally announced May 2024.

Comments: CVPR 2024

Transcriptomics-guided Slide Representation Learning in Computational Pathology

Authors: Guillaume Jaume, Lukas Oldenburg, Anurag Vaidya, Richard J. Chen, Drew F. K. Williamson, Thomas Peeters, Andrew H. Song, Faisal Mahmood

Abstract: Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels), but scaling these models to learn slide embeddings from the entirety of giga-pixel whole-slide images (WSIs) remains challenging. Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-traini… ▽ More Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels), but scaling these models to learn slide embeddings from the entirety of giga-pixel whole-slide images (WSIs) remains challenging. Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-training. Expression profiles constitute highly detailed molecular descriptions of a tissue that we hypothesize offer a strong task-agnostic training signal for learning slide embeddings. Our slide and expression (S+E) pre-training strategy, called Tangle, employs modality-specific encoders, the outputs of which are aligned via contrastive learning. Tangle was pre-trained on samples from three different organs: liver (n=6,597 S+E pairs), breast (n=1,020), and lung (n=1,012) from two different species (Homo sapiens and Rattus norvegicus). Across three independent test datasets consisting of 1,265 breast WSIs, 1,946 lung WSIs, and 4,584 liver WSIs, Tangle shows significantly better few-shot performance compared to supervised and SSL baselines. When assessed using prototype-based classification and slide retrieval, Tangle also shows a substantial performance improvement over all baselines. Code available at https://github.com/mahmoodlab/TANGLE. △ Less

Submitted 19 May, 2024; originally announced May 2024.

Comments: CVPR'24, Oral

Artificial Intelligence for Digital and Computational Pathology

Authors: Andrew H. Song, Guillaume Jaume, Drew F. K. Williamson, Ming Y. Lu, Anurag Vaidya, Tiffany R. Miller, Faisal Mahmood

Abstract: Advances in digitizing tissue slides and the fast-paced progress in artificial intelligence, including deep learning, have boosted the field of computational pathology. This field holds tremendous potential to automate clinical diagnosis, predict patient prognosis and response to therapy, and discover new morphological biomarkers from tissue images. Some of these artificial intelligence-based syst… ▽ More Advances in digitizing tissue slides and the fast-paced progress in artificial intelligence, including deep learning, have boosted the field of computational pathology. This field holds tremendous potential to automate clinical diagnosis, predict patient prognosis and response to therapy, and discover new morphological biomarkers from tissue images. Some of these artificial intelligence-based systems are now getting approved to assist clinical diagnosis; however, technical barriers remain for their widespread clinical adoption and integration as a research tool. This Review consolidates recent methodological advances in computational pathology for predicting clinical end points in whole-slide images and highlights how these developments enable the automation of clinical practice and the discovery of new biomarkers. We then provide future perspectives as the field expands into a broader range of clinical and research tasks with increasingly diverse modalities of clinical data. △ Less

Submitted 12 December, 2023; originally announced January 2024.

Journal ref: Nature Reviews Bioengineering 2023

A Foundational Multimodal Vision Language AI Assistant for Human Pathology

Authors: Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Kenji Ikamura, Georg Gerber, Ivy Liang, Long Phi Le, Tong Ding, Anil V Parwani, Faisal Mahmood

Abstract: The field of computational pathology has witnessed remarkable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders. However, despite the explosive growth of generative artificial intelligence (AI), there has been limited study on building general purpose, multimodal AI assistants tailored to pathology. Here we present PathChat, a vis… ▽ More The field of computational pathology has witnessed remarkable progress in the development of both task-specific predictive models and task-agnostic self-supervised vision encoders. However, despite the explosive growth of generative artificial intelligence (AI), there has been limited study on building general purpose, multimodal AI assistants tailored to pathology. Here we present PathChat, a vision-language generalist AI assistant for human pathology using an in-house developed foundational vision encoder pretrained on 100 million histology images from over 100,000 patient cases and 1.18 million pathology image-caption pairs. The vision encoder is then combined with a pretrained large language model and the whole system is finetuned on over 250,000 diverse disease agnostic visual language instructions. We compare PathChat against several multimodal vision language AI assistants as well as GPT4V, which powers the commercially available multimodal general purpose AI assistant ChatGPT-4. When relevant clinical context is provided with the histology image, PathChat achieved a diagnostic accuracy of 87% on multiple-choice questions based on publicly available cases of diverse tissue origins and disease models. Additionally, using open-ended questions and human expert evaluation, we found that overall PathChat produced more accurate and pathologist-preferable responses to diverse queries related to pathology. As an interactive and general vision language AI assistant that can flexibly handle both visual and natural language inputs, PathChat can potentially find impactful applications in pathology education, research, and human-in-the-loop clinical decision making. △ Less

Submitted 12 December, 2023; originally announced December 2023.

A General-Purpose Self-Supervised Model for Computational Pathology

Authors: Richard J. Chen, Tong Ding, Ming Y. Lu, Drew F. K. Williamson, Guillaume Jaume, Bowen Chen, Andrew Zhang, Daniel Shao, Andrew H. Song, Muhammad Shaban, Mane Williams, Anurag Vaidya, Sharifa Sahai, Lukas Oldenburg, Luca L. Weishaupt, Judy J. Wang, Walt Williams, Long Phi Le, Georg Gerber, Faisal Mahmood

Abstract: Tissue phenoty** is a fundamental computational pathology (CPath) task in learning objective characterizations of histopathologic biomarkers in anatomic pathology. However, whole-slide imaging (WSI) poses a complex computer vision problem in which the large-scale image resolutions of WSIs and the enormous diversity of morphological phenotypes preclude large-scale data annotation. Current efforts… ▽ More Tissue phenoty** is a fundamental computational pathology (CPath) task in learning objective characterizations of histopathologic biomarkers in anatomic pathology. However, whole-slide imaging (WSI) poses a complex computer vision problem in which the large-scale image resolutions of WSIs and the enormous diversity of morphological phenotypes preclude large-scale data annotation. Current efforts have proposed using pretrained image encoders with either transfer learning from natural image datasets or self-supervised pretraining on publicly-available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using over 100 million tissue patches from over 100,000 diagnostic haematoxylin and eosin-stained WSIs across 20 major tissue types, and evaluated on 33 representative CPath clinical tasks in CPath of varying diagnostic difficulties. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subty** generalization in classifying up to 108 cancer types in the OncoTree code classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient AI models that can generalize and transfer to a gamut of diagnostically-challenging tasks and clinical workflows in anatomic pathology. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples

Authors: Andrew H. Song, Mane Williams, Drew F. K. Williamson, Guillaume Jaume, Andrew Zhang, Bowen Chen, Robert Serafin, Jonathan T. C. Liu, Alex Baras, Anil V. Parwani, Faisal Mahmood

Abstract: Human tissue and its constituent cells form a microenvironment that is fundamentally three-dimensional (3D). However, the standard-of-care in pathologic diagnosis involves selecting a few two-dimensional (2D) sections for microscopic evaluation, risking sampling bias and misdiagnosis. Diverse methods for capturing 3D tissue morphologies have been developed, but they have yet had little translation… ▽ More Human tissue and its constituent cells form a microenvironment that is fundamentally three-dimensional (3D). However, the standard-of-care in pathologic diagnosis involves selecting a few two-dimensional (2D) sections for microscopic evaluation, risking sampling bias and misdiagnosis. Diverse methods for capturing 3D tissue morphologies have been developed, but they have yet had little translation to clinical practice; manual and computational evaluations of such large 3D data have so far been impractical and/or unable to provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy or microcomputed tomography and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting the value of capturing larger extents of heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response. △ Less

Submitted 27 July, 2023; originally announced July 2023.

Towards a Visual-Language Foundation Model for Computational Pathology

Authors: Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, Georg Gerber, Anil V Parwani, Faisal Mahmood

Abstract: The accelerated adoption of digital pathology and advances in deep learning have enabled the development of powerful models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain and the model's usage is limited by the specific task and disease for which it is trained. Additionally,… ▽ More The accelerated adoption of digital pathology and advances in deep learning have enabled the development of powerful models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain and the model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text, and notably over 1.17 million image-caption pairs via task-agnostic pretraining. Evaluated on a suite of 13 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving either or both histopathology images and text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning. △ Less

Submitted 25 July, 2023; v1 submitted 24 July, 2023; originally announced July 2023.

Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images

Authors: Ming Y. Lu, Bowen Chen, Andrew Zhang, Drew F. K. Williamson, Richard J. Chen, Tong Ding, Long Phi Le, Yung-Sung Chuang, Faisal Mahmood

Abstract: Contrastive visual language pretraining has emerged as a powerful method for either training new language-aware image encoders or augmenting existing pretrained models with zero-shot visual recognition capabilities. However, existing works typically train on large datasets of image-text pairs and have been designed to perform downstream tasks involving only small to medium sized-images, neither of… ▽ More Contrastive visual language pretraining has emerged as a powerful method for either training new language-aware image encoders or augmenting existing pretrained models with zero-shot visual recognition capabilities. However, existing works typically train on large datasets of image-text pairs and have been designed to perform downstream tasks involving only small to medium sized-images, neither of which are applicable to the emerging field of computational pathology where there are limited publicly available paired image-text datasets and each image can span up to 100,000 x 100,000 pixels. In this paper we present MI-Zero, a simple and intuitive framework for unleashing the zero-shot transfer capabilities of contrastively aligned image and text models on gigapixel histopathology whole slide images, enabling multiple downstream diagnostic tasks to be carried out by pretrained encoders without requiring any additional labels. MI-Zero reformulates zero-shot transfer under the framework of multiple instance learning to overcome the computational challenge of inference on extremely large images. We used over 550k pathology reports and other available in-domain text corpora to pre-train our text encoder. By effectively leveraging strong pre-trained encoders, our best model pretrained on over 33k histopathology image-caption pairs achieves an average median zero-shot accuracy of 70.2% across three different real-world cancer subty** tasks. Our code is available at: https://github.com/mahmoodlab/MI-Zero. △ Less

Submitted 13 June, 2023; originally announced June 2023.

Comments: Accepted to CVPR 2023

Incorporating intratumoral heterogeneity into weakly-supervised deep learning models via variance pooling

Authors: Iain Carmichael, Andrew H. Song, Richard J. Chen, Drew F. K. Williamson, Tiffany Y. Chen, Faisal Mahmood

Abstract: Supervised learning tasks such as cancer survival prediction from gigapixel whole slide images (WSIs) are a critical challenge in computational pathology that requires modeling complex features of the tumor microenvironment. These learning tasks are often solved with deep multi-instance learning (MIL) models that do not explicitly capture intratumoral heterogeneity. We develop a novel variance poo… ▽ More Supervised learning tasks such as cancer survival prediction from gigapixel whole slide images (WSIs) are a critical challenge in computational pathology that requires modeling complex features of the tumor microenvironment. These learning tasks are often solved with deep multi-instance learning (MIL) models that do not explicitly capture intratumoral heterogeneity. We develop a novel variance pooling architecture that enables a MIL model to incorporate intratumoral heterogeneity into its predictions. Two interpretability tools based on representative patches are illustrated to probe the biological signals captured by these models. An empirical study with 4,479 gigapixel WSIs from the Cancer Genome Atlas shows that adding variance pooling onto MIL frameworks improves survival prediction performance for five cancer types. △ Less

Submitted 19 November, 2022; v1 submitted 17 June, 2022; originally announced June 2022.

Comments: MICCAI 2022

Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images

Authors: Kevin Thandiackal, Boqi Chen, Pushpak Pati, Guillaume Jaume, Drew F. K. Williamson, Maria Gabrani, Orcun Goksel

Abstract: Multiple Instance Learning (MIL) methods have become increasingly popular for classifying giga-pixel sized Whole-Slide Images (WSIs) in digital pathology. Most MIL methods operate at a single WSI magnification, by processing all the tissue patches. Such a formulation induces high computational requirements, and constrains the contextualization of the WSI-level representation to a single scale. A f… ▽ More Multiple Instance Learning (MIL) methods have become increasingly popular for classifying giga-pixel sized Whole-Slide Images (WSIs) in digital pathology. Most MIL methods operate at a single WSI magnification, by processing all the tissue patches. Such a formulation induces high computational requirements, and constrains the contextualization of the WSI-level representation to a single scale. A few MIL methods extend to multiple scales, but are computationally more demanding. In this paper, inspired by the pathological diagnostic process, we propose ZoomMIL, a method that learns to perform multi-level zooming in an end-to-end manner. ZoomMIL builds WSI representations by aggregating tissue-context information from multiple magnifications. The proposed method outperforms the state-of-the-art MIL methods in WSI classification on two large datasets, while significantly reducing the computational demands with regard to Floating-Point Operations (FLOPs) and processing time by up to 40x. △ Less

Submitted 26 July, 2022; v1 submitted 26 April, 2022; originally announced April 2022.

Comments: Typos corrected; Changed dataset name from INSEC to CRC upon dataset creators' request; Update affiliation and fix typos;

Emission Line Variability during a Nonthermal Outburst in the Gamma-Ray Bright Quasar 1156+295

Authors: Melissa K. Hallum, Svetlana G. Jorstad, Valeri M. Larionov, Alan P. Marscher, Manasvita Joshi, Zachary R. Weaver, Karen E. Williamson, Ivan Agudo, George A. Borman, Carolina Casadio, Antonio Fuentes, Tatiana S. Grishina, Evgenia N. Kopatskaya, Elena G. Larionova, Liyudmila V. Larionova, Daria A. Morozova, Anna A. Nikiforova, Sergey S. Savchenko, Ivan S. Troitsky, Yulia V. Troitskaya, Andrey A. Vasilyev

Abstract: We present multi-epoch optical spectra of the $γ$-ray bright blazar 1156+295 (4C +29.45, Ton 599) obtained with the 4.3~m Lowell Discovery Telescope. During a multi-wavelength outburst in late 2017, when the $γ$-ray flux increased to $2.5\times 10^{-6} \; \rm phot\; cm^{-2}\; s^{-1}$ and the quasar was first detected at energies $\geq100$ GeV, the flux of the Mg II $λ2798$ emission line changed, a… ▽ More We present multi-epoch optical spectra of the $γ$-ray bright blazar 1156+295 (4C +29.45, Ton 599) obtained with the 4.3~m Lowell Discovery Telescope. During a multi-wavelength outburst in late 2017, when the $γ$-ray flux increased to $2.5\times 10^{-6} \; \rm phot\; cm^{-2}\; s^{-1}$ and the quasar was first detected at energies $\geq100$ GeV, the flux of the Mg II $λ2798$ emission line changed, as did that of the Fe emission complex at shorter wavelengths. These emission line fluxes increased along with the highly polarized optical continuum flux, which is presumably synchrotron radiation from the relativistic jet, with a relative time delay of $\lesssim2$ weeks. This implies that the line-emitting clouds lie near the jet, which points almost directly toward the line of sight. The emission-line radiation from such clouds, which are located outside the canonical accretion-disk related broad-line region, may be a primary source of seed photons that are up-scattered to $γ$-ray energies by relativistic electrons in the jet. △ Less

Submitted 31 January, 2022; originally announced February 2022.

Comments: 25 pages, 12 figures

Journal ref: 2022 ApJ 926 180

Radio and $γ$-ray activity in the jet of the blazar S5 0716$+$714

Authors: Dae-Won Kim, Evgeniya V. Kravchenko, Alexander M. Kutkin, Markus Böttcher, José L. Gómez, Mark Gurwell, Svetlana G. Jorstad, Anne Lähteenmäki, Alan P. Marscher, Venkatessh Ramakrishnan, Merja Tornikoski, Sascha Trippe, Zachary Weaver, Karen E. Williamson

Abstract: We explore the connection between the $γ$-ray and radio emission in the jet of the blazar 0716$+$714 by using 15, 37, and 230 GHz radio and 0.1$-$200 GeV $γ$-ray light curves spanning 10.5 years (2008$-$2019). We find significant positive and negative correlations between radio and $γ$-ray fluxes in different time ranges. The time delays between radio and $γ$-ray emission suggest that the observed… ▽ More We explore the connection between the $γ$-ray and radio emission in the jet of the blazar 0716$+$714 by using 15, 37, and 230 GHz radio and 0.1$-$200 GeV $γ$-ray light curves spanning 10.5 years (2008$-$2019). We find significant positive and negative correlations between radio and $γ$-ray fluxes in different time ranges. The time delays between radio and $γ$-ray emission suggest that the observed $γ$-ray flares originated from multiple regions upstream of the radio core, within a few parsecs from the central engine. Using time-resolved 43 GHz VLBA maps we identified 14 jet components moving downstream along the jet. Their apparent speeds range from 6 to 26 $c$, showing notable variations in their position angles upstream the stationary component ($\sim$0.53 mas from the core). The brightness temperature declines as function of distance from the core according to a power-law which becomes shallower at the location of the stationary component. We also find that the periods at which significant correlations between radio and $γ$-ray emission occur overlap with the times when the jet was oriented to the north. Our results indicate that the passage of a propagating disturbance (or shock) through the radio core and the orientation of the jet might be responsible for the observed correlation between the radio and $γ$-ray variability. We present a scenario that connects the positive correlation and the unusual anti-correlation by combining the production of a flare and a dip at $γ$-rays by a strong moving shock at different distances from the jet apex. △ Less

Submitted 4 November, 2021; originally announced November 2021.

Comments: 26 pages, 13 figures, 5 tables, Accepted for publication in ApJ

Algorithm Fairness in AI for Medicine and Healthcare

Authors: Richard J. Chen, Tiffany Y. Chen, Jana Lipkova, Judy J. Wang, Drew F. K. Williamson, Ming Y. Lu, Sharifa Sahai, Faisal Mahmood

Abstract: In the current development and deployment of many artificial intelligence (AI) systems in healthcare, algorithm fairness is a challenging problem in delivering equitable care. Recent evaluation of AI models stratified across race sub-populations have revealed inequalities in how patients are diagnosed, given treatments, and billed for healthcare costs. In this perspective article, we summarize the… ▽ More In the current development and deployment of many artificial intelligence (AI) systems in healthcare, algorithm fairness is a challenging problem in delivering equitable care. Recent evaluation of AI models stratified across race sub-populations have revealed inequalities in how patients are diagnosed, given treatments, and billed for healthcare costs. In this perspective article, we summarize the intersectional field of fairness in machine learning through the context of current issues in healthcare, outline how algorithmic biases (e.g. - image acquisition, genetic variation, intra-observer labeling variability) arise in current clinical workflows and their resulting healthcare disparities. Lastly, we also review emerging technology for mitigating bias via federated learning, disentanglement, and model explainability, and their role in AI-SaMD development. △ Less

Submitted 23 March, 2022; v1 submitted 1 October, 2021; originally announced October 2021.

Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning

Authors: Richard J. Chen, Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Jana Lipkova, Muhammad Shaban, Maha Shady, Mane Williams, Bum** Joo, Zahra Noor, Faisal Mahmood

Abstract: The rapidly emerging field of deep learning-based computational pathology has demonstrated promise in develo** objective prognostic models from histology whole slide images. However, most prognostic models are either based on histology or genomics alone and do not address how histology and genomics can be integrated to develop joint image-omic prognostic models. Additionally identifying explaina… ▽ More The rapidly emerging field of deep learning-based computational pathology has demonstrated promise in develo** objective prognostic models from histology whole slide images. However, most prognostic models are either based on histology or genomics alone and do not address how histology and genomics can be integrated to develop joint image-omic prognostic models. Additionally identifying explainable morphological and molecular descriptors from these models that govern such prognosis is of interest. We used multimodal deep learning to integrate gigapixel whole slide pathology images, RNA-seq abundance, copy number variation, and mutation data from 5,720 patients across 14 major cancer types. Our interpretable, weakly-supervised, multimodal deep learning algorithm is able to fuse these heterogeneous modalities for predicting outcomes and discover prognostic features from these modalities that corroborate with poor and favorable outcomes via multimodal interpretability. We compared our model with unimodal deep learning models trained on histology slides and molecular profiles alone, and demonstrate performance increase in risk stratification on 9 out of 14 cancers. In addition, we analyze morphologic and molecular markers responsible for prognostic predictions across all cancer types. All analyzed data, including morphological and molecular correlates of patient prognosis across the 14 cancer types at a disease and patient level are presented in an interactive open-access database (http://pancancer.mahmoodlab.org) to allow for further exploration and prognostic biomarker discovery. To validate that these model explanations are prognostic, we further analyzed high attention morphological regions in WSIs, which indicates that tumor-infiltrating lymphocyte presence corroborates with favorable cancer prognosis on 9 out of 14 cancer types studied. △ Less

Submitted 4 August, 2021; originally announced August 2021.

Comments: Demo: http://pancancer.mahmoodlab.org

Fast and Scalable Image Search For Histology

Authors: Chengkuan Chen, Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Andrew J. Schaumberg, Faisal Mahmood

Abstract: The expanding adoption of digital pathology has enabled the curation of large repositories of histology whole slide images (WSIs), which contain a wealth of information. Similar pathology image search offers the opportunity to comb through large historical repositories of gigapixel WSIs to identify cases with similar morphological features and can be particularly useful for diagnosing rare disease… ▽ More The expanding adoption of digital pathology has enabled the curation of large repositories of histology whole slide images (WSIs), which contain a wealth of information. Similar pathology image search offers the opportunity to comb through large historical repositories of gigapixel WSIs to identify cases with similar morphological features and can be particularly useful for diagnosing rare diseases, identifying similar cases for predicting prognosis, treatment outcomes, and potential clinical trial success. A critical challenge in develo** a WSI search and retrieval system is scalability, which is uniquely challenging given the need to search a growing number of slides that each can consist of billions of pixels and are several gigabytes in size. Such systems are typically slow and retrieval speed often scales with the size of the repository they search through, making their clinical adoption tedious and are not feasible for repositories that are constantly growing. Here we present Fast Image Search for Histopathology (FISH), a histology image search pipeline that is infinitely scalable and achieves constant search speed that is independent of the image database size while being interpretable and without requiring detailed annotations. FISH uses self-supervised deep learning to encode meaningful representations from WSIs and a Van Emde Boas tree for fast search, followed by an uncertainty-based ranking algorithm to retrieve similar WSIs. We evaluated FISH on multiple tasks and datasets with over 22,000 patient cases spanning 56 disease subtypes. We additionally demonstrate that FISH can be used to assist with the diagnosis of rare cancer types where sufficient cases may not be available to train traditional supervised deep models. FISH is available as an easy-to-use, open-source software package (https://github.com/mahmoodlab/FISH). △ Less

Submitted 28 July, 2021; originally announced July 2021.

Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks

Authors: Richard J. Chen, Ming Y. Lu, Muhammad Shaban, Chengkuan Chen, Tiffany Y. Chen, Drew F. K. Williamson, Faisal Mahmood

Abstract: Cancer prognostication is a challenging task in computational pathology that requires context-aware representations of histology features to adequately infer patient survival. Despite the advancements made in weakly-supervised deep learning, many approaches are not context-aware and are unable to model important morphological feature interactions between cell identities and tissue types that are p… ▽ More Cancer prognostication is a challenging task in computational pathology that requires context-aware representations of histology features to adequately infer patient survival. Despite the advancements made in weakly-supervised deep learning, many approaches are not context-aware and are unable to model important morphological feature interactions between cell identities and tissue types that are prognostic for patient survival. In this work, we present Patch-GCN, a context-aware, spatially-resolved patch-based graph convolutional network that hierarchically aggregates instance-level histology features to model local- and global-level topological structures in the tumor microenvironment. We validate Patch-GCN with 4,370 gigapixel WSIs across five different cancer types from the Cancer Genome Atlas (TCGA), and demonstrate that Patch-GCN outperforms all prior weakly-supervised approaches by 3.58-9.46%. Our code and corresponding models are publicly available at https://github.com/mahmoodlab/Patch-GCN. △ Less

Submitted 27 July, 2021; originally announced July 2021.

Comments: MICCAI 2021

Deep Learning-based Frozen Section to FFPE Translation

Authors: Kutsev Bengisu Ozyoruk, Sermet Can, Guliz Irem Gokceler, Kayhan Basak, Derya Demir, Gurdeniz Serin, Uguray Payam Hacisalihoglu, Emirhan Kurtuluş, Berkan Darbaz, Ming Y. Lu, Tiffany Y. Chen, Drew F. K. Williamson, Funda Yilmaz, Faisal Mahmood, Mehmet Turan

Abstract: Frozen sectioning (FS) is the preparation method of choice for microscopic evaluation of tissues during surgical operations. The high speed of the procedure allows pathologists to rapidly assess the key microscopic features, such as tumour margins and malignant status to guide surgical decision-making and minimise disruptions to the course of the operation. However, FS is prone to introducing many… ▽ More Frozen sectioning (FS) is the preparation method of choice for microscopic evaluation of tissues during surgical operations. The high speed of the procedure allows pathologists to rapidly assess the key microscopic features, such as tumour margins and malignant status to guide surgical decision-making and minimise disruptions to the course of the operation. However, FS is prone to introducing many misleading artificial structures (histological artefacts), such as nuclear ice crystals, compression, and cutting artefacts, hindering timely and accurate diagnostic judgement of the pathologist. Additional training and prolonged experience is often required to make highly effective and time-critical diagnosis on frozen sections. On the other hand, the gold standard tissue preparation technique of formalin-fixation and paraffin-embedding (FFPE) provides significantly superior image quality, but is a very time-consuming process (12-48 hours), making it unsuitable for intra-operative use. In this paper, we propose an artificial intelligence (AI) method that improves FS image quality by computationally transforming frozen-sectioned whole-slide images (FS-WSIs) into whole-slide FFPE-style images in minutes. AI-FFPE rectifies FS artefacts with the guidance of an attention mechanism that puts a particular emphasis on artefacts while utilising a self-regularization mechanism established between FS input image and synthesized FFPE-style image that preserves clinically relevant features. As a result, AI-FFPE method successfully generates FFPE-style images without significantly extending tissue processing time and consequently improves diagnostic accuracy. We demonstrate the efficacy of AI-FFPE on lung and brain frozen sections using a variety of different qualitative and quantitative metrics including visual Turing tests from 20 board certified pathologists. △ Less

Submitted 2 November, 2021; v1 submitted 25 July, 2021; originally announced July 2021.

Broadband Multi-wavelength Properties of M87 during the 2017 Event Horizon Telescope Campaign

Authors: J. C. Algaba, J. Anczarski, K. Asada, M. Balokovic, S. Chandra, Y. -Z. Cui, A. D. Falcone, M. Giroletti, C. Goddi, K. Hada, D. Haggard, S. Jorstad, A. Kaur, T. Kawashima, G. Keating, J. -Y. Kim, M. Kino, S. Komossa, E. V. Kravchenko, T. P. Krichbaum, S. -S. Lee, R. -S. Lu, M. Lucchini, S. Markoff, J. Neilsen , et al. (14 additional authors not shown)

Abstract: In 2017, the Event Horizon Telescope (EHT) Collaboration succeeded in capturing the first direct image of the center of the M87 galaxy. The asymmetric ring morphology and size are consistent with theoretical expectations for a weakly accreting supermassive black hole of mass approximately 6.5 x 10^9 M_solar. The EHTC also partnered with several international facilities in space and on the ground,… ▽ More In 2017, the Event Horizon Telescope (EHT) Collaboration succeeded in capturing the first direct image of the center of the M87 galaxy. The asymmetric ring morphology and size are consistent with theoretical expectations for a weakly accreting supermassive black hole of mass approximately 6.5 x 10^9 M_solar. The EHTC also partnered with several international facilities in space and on the ground, to arrange an extensive, quasi-simultaneous multi-wavelength campaign. This Letter presents the results and analysis of this campaign, as well as the multi-wavelength data as a legacy data repository. We captured M87 in a historically low state, and the core flux dominates over HST-1 at high energies, making it possible to combine core flux constraints with the more spatially precise very long baseline interferometry data. We present the most complete simultaneous multi-wavelength spectrum of the active nucleus to date, and discuss the complexity and caveats of combining data from different spatial scales into one broadband spectrum. We apply two heuristic, isotropic leptonic single-zone models to provide insight into the basic source properties, but conclude that a structured jet is necessary to explain M87's spectrum. We can exclude that the simultaneous gamma-ray emission is produced via inverse Compton emission in the same region producing the EHT mm-band emission, and further conclude that the gamma-rays can only be produced in the inner jets (inward of HST-1) if there are strongly particle-dominated regions. Direct synchrotron emission from accelerated protons and secondaries cannot yet be excluded. △ Less

Submitted 14 April, 2021; originally announced April 2021.

Comments: 43 pages, 19 figures, 10 tables; complete author list available in manuscript; The Astrophysical Journal Letters, 2021, 911, L11; publication doi: 3847/2041-8213/abef71, data doi: 10.25739/mhh2-cw46

Application of adaptive ANOVA and reduced basis methods to the stochastic Stokes-Brinkman problem

Authors: Kevin Williamson, Heyrim Cho, Bedřich Sousedík

Abstract: The Stokes-Brinkman equations model fluid flow in highly heterogeneous porous media. In this paper, we consider the numerical solution of the Stokes-Brinkman equations with stochastic permeabilities, where the permeabilities in subdomains are assumed to be independent and uniformly distributed within a known interval. We employ a truncated anchored ANOVA decomposition alongside stochastic collocat… ▽ More The Stokes-Brinkman equations model fluid flow in highly heterogeneous porous media. In this paper, we consider the numerical solution of the Stokes-Brinkman equations with stochastic permeabilities, where the permeabilities in subdomains are assumed to be independent and uniformly distributed within a known interval. We employ a truncated anchored ANOVA decomposition alongside stochastic collocation to estimate the moments of the velocity and pressure solutions. Through an adaptive procedure selecting only the most important ANOVA directions, we reduce the number of collocation points needed for accurate estimation of the statistical moments. However, for even modest stochastic dimensions, the number of collocation points remains too large to perform high-fidelity solves at each point. We use reduced basis methods to alleviate the computational burden by approximating the expensive high-fidelity solves with inexpensive approximate solutions on a low-dimensional space. We furthermore develop and analyze rigorous a posteriori error estimates for the reduced basis approximation. We apply these methods to 2D problems considering both isotropic and anisotropic permeabilities. △ Less

Submitted 13 February, 2021; originally announced February 2021.

Comments: 20 pages, 12 figures

MSC Class: 35R60; 65C30; 60H35; 65N75

Journal ref: Computational Geosciences, 25, 1191-1213, 2021

Space Public Outreach Team: Successful STEM Engagement on Complex Technical Topics

Authors: Angela Des Jardins, Joey Shapiro Key, Kathryn Williamson, Seth Kimbrell, Sophie De Saint-Georges, Tyson Littenberg, Jessica Page, Timothy Dolch

Abstract: It is the responsibility of today's scientists, engineers, and educators to inspire and encourage our youth into technical careers that benefit our society. Too often, however, this responsibility is buried beneath daily job demands and the routines of teaching. Space Public Outreach Team (SPOT) programs leverage a train-the-trainer model to empower college students to make meaningful impacts in t… ▽ More It is the responsibility of today's scientists, engineers, and educators to inspire and encourage our youth into technical careers that benefit our society. Too often, however, this responsibility is buried beneath daily job demands and the routines of teaching. Space Public Outreach Team (SPOT) programs leverage a train-the-trainer model to empower college students to make meaningful impacts in their local communities by engaging and inspiring younger students through science presentations. SPOT takes advantage of the excitement of space and the natural way college students serve as role models for children. The result is a win-win program for all involved. This paper describes the original Montana SPOT program, presents analyses demonstrating the success of SPOT, gives overviews of program adaptations in West Virginia and with the NANOGrav collaboration, describes how college student presenters are able to share complex topics, and discusses the importance of college student role models. We hope that our experiences with SPOT will help others implement similar strategies in their own communities. △ Less

Submitted 29 October, 2020; originally announced October 2020.

Comments: 21 pages, 10 figures

Journal ref: JCMST Volume 39, Number 4, 2020, 339-359

Federated Learning for Computational Pathology on Gigapixel Whole Slide Images

Authors: Ming Y. Lu, Dehan Kong, Jana Lipkova, Richard J. Chen, Rajendra Singh, Drew F. K. Williamson, Tiffany Y. Chen, Faisal Mahmood

Abstract: Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non-human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable, and accurate deep learning-based models often rely on the collect… ▽ More Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non-human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable, and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns amongst other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation. △ Less

Submitted 22 September, 2020; v1 submitted 21 September, 2020; originally announced September 2020.

Multi-Wavelength Variability of BL Lacertae Measured with High Time Resolution

Authors: Zachary R. Weaver, K. E. Williamson, S. G. Jorstad, A. P. Marscher, V. M. Larionov, C. M. Raiteri, M. Villata, J. A. Acosta-Pulido, R. Bachev, G. V. Baida, T. J. Balonek, E. Benitez, G. A. Borman, V. Bozhilov, M. I. Carnerero, D. Carosati, W. P. Chen, G. Damljanovic, V. Dhiman, D. J. Dougherty, S. A. Ehgamberdiev, T. S. Grishina, A. C. Gupta, M. Hart, D. Hiriart , et al. (32 additional authors not shown)

Abstract: In an effort to locate the sites of emission at different frequencies and physical processes causing variability in blazar jets, we have obtained high time-resolution observations of BL Lacertae over a wide wavelength range: with the \emph{Transiting Exoplanet Survey Satellite} (TESS) at 6,000-10,000 Å with 2-minute cadence; with the Neil Gehrels \emph{Swift} satellite at optical, UV, and X-ray ba… ▽ More In an effort to locate the sites of emission at different frequencies and physical processes causing variability in blazar jets, we have obtained high time-resolution observations of BL Lacertae over a wide wavelength range: with the \emph{Transiting Exoplanet Survey Satellite} (TESS) at 6,000-10,000 Å with 2-minute cadence; with the Neil Gehrels \emph{Swift} satellite at optical, UV, and X-ray bands; with the Nuclear Spectroscopic Telescope Array at hard X-ray bands; with the \emph{Fermi} Large Area Telescope at $γ$-ray energies; and with the Whole Earth Blazar Telescope for measurement of the optical flux density and polarization. All light curves are correlated, with similar structure on timescales from hours to days. The shortest timescale of variability at optical frequencies observed with TESS is $\sim 0.5$ hr. The most common timescale is $13\pm1$~hr, comparable with the minimum timescale of X-ray variability, 14.5 hr. The multi-wavelength variability properties cannot be explained by a change solely in the Doppler factor of the emitting plasma. The polarization behavior implies that there are both ordered and turbulent components to the magnetic field in the jet. Correlation analysis indicates that the X-ray variations lag behind the $γ$-ray and optical light curves by up to $\sim 0.4$ days. The timescales of variability, cross-frequency lags, and polarization properties can be explained by turbulent plasma that is energized by a shock in the jet and subsequently loses energy to synchrotron and inverse Compton radiation in a magnetic field of strength $\sim3$ G △ Less

Submitted 15 July, 2020; originally announced July 2020.

Comments: 33 pages, 25 figures, 14 tables. Accepted to ApJ

Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary

Authors: Ming Y. Lu, Melissa Zhao, Maha Shady, Jana Lipkova, Tiffany Y. Chen, Drew F. K. Williamson, Faisal Mahmood

Abstract: Cancer of unknown primary (CUP) is an enigmatic group of diagnoses where the primary anatomical site of tumor origin cannot be determined. This poses a significant challenge since modern therapeutics such as chemotherapy regimen and immune checkpoint inhibitors are specific to the primary tumor. Recent work has focused on using genomics and transcriptomics for identification of tumor origins. Howe… ▽ More Cancer of unknown primary (CUP) is an enigmatic group of diagnoses where the primary anatomical site of tumor origin cannot be determined. This poses a significant challenge since modern therapeutics such as chemotherapy regimen and immune checkpoint inhibitors are specific to the primary tumor. Recent work has focused on using genomics and transcriptomics for identification of tumor origins. However, genomic testing is not conducted for every patient and lacks clinical penetration in low resource settings. Herein, to overcome these challenges, we present a deep learning-based computational pathology algorithm-TOAD-that can provide a differential diagnosis for CUP using routinely acquired histology slides. We used 17,486 gigapixel whole slide images with known primaries spread over 18 common origins to train a multi-task deep model to simultaneously identify the tumor as primary or metastatic and predict its site of origin. We tested our model on an internal test set of 4,932 cases with known primaries and achieved a top-1 accuracy of 0.84, a top-3 accuracy of 0.94 while on our external test set of 662 cases from 202 different hospitals, it achieved a top-1 and top-3 accuracy of 0.79 and 0.93 respectively. We further curated a dataset of 717 CUP cases from 151 different medical centers and identified a subset of 290 cases for which a differential diagnosis was assigned. Our model predictions resulted in concordance for 50% of cases (\k{appa}=0.4 when adjusted for agreement by chance) and a top-3 agreement of 75%. Our proposed method can be used as an assistive tool to assign differential diagnosis to complicated metastatic and CUP cases and could be used in conjunction with or in lieu of immunohistochemical analysis and extensive diagnostic work-ups to reduce the occurrence of CUP. △ Less

Submitted 28 June, 2020; v1 submitted 24 June, 2020; originally announced June 2020.

Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images

Authors: Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri, Faisal Mahmood

Abstract: The rapidly emerging field of computational pathology has the potential to enable objective diagnosis, therapeutic response prediction and identification of new morphological features of clinical relevance. However, deep learning-based computational pathology approaches either require manual annotation of gigapixel whole slide images (WSIs) in fully-supervised settings or thousands of WSIs with sl… ▽ More The rapidly emerging field of computational pathology has the potential to enable objective diagnosis, therapeutic response prediction and identification of new morphological features of clinical relevance. However, deep learning-based computational pathology approaches either require manual annotation of gigapixel whole slide images (WSIs) in fully-supervised settings or thousands of WSIs with slide-level labels in a weakly-supervised setting. Moreover, whole slide level computational pathology methods also suffer from domain adaptation and interpretability issues. These challenges have prevented the broad adaptation of computational pathology for clinical and research purposes. Here we present CLAM - Clustering-constrained attention multiple instance learning, an easy-to-use, high-throughput, and interpretable WSI-level processing and learning method that only requires slide-level labels while being data efficient, adaptable and capable of handling multi-class subty** problems. CLAM is a deep-learning-based weakly-supervised method that uses attention-based learning to automatically identify sub-regions of high diagnostic value in order to accurately classify the whole slide, while also utilizing instance-level clustering over the representative regions identified to constrain and refine the feature space. In three separate analyses, we demonstrate the data efficiency and adaptability of CLAM and its superior performance over standard weakly-supervised classification. We demonstrate that CLAM models are interpretable and can be used to identify well-known and new morphological features. We further show that models trained using CLAM are adaptable to independent test cohorts, cell phone microscopy images, and biopsies. CLAM is a general-purpose and adaptable method that can be used for a variety of different computational pathology tasks in both clinical and research settings. △ Less

Submitted 21 May, 2020; v1 submitted 20 April, 2020; originally announced April 2020.

Multiwavelength behaviour of the blazar 3C279: decade-long study from $γ$-ray to radio

Authors: V. M. Larionov, S. G. Jorstad, A. P. Marscher, M. Villata, C. M. Raiteri, P. S. Smith, I. Agudo, S. S. Savchenko, D. A. Morozova, J. A. Acosta-Pulido, M. F. Aller, H. D. Aller, T. S. Andreeva, A. A. Arkharov, R. Bachev, G. Bonnoli, G. A. Borman, V. Bozhilov, P. Calcidese, M. I. Carnerero, D. Carosati, C. Casadio, W. -P. Chen, G. Damljanovic, A. V. Dementyev , et al. (62 additional authors not shown)

Abstract: We report the results of decade-long (2008-2018) $γ$-ray to 1 GHz radio monitoring of the blazar 3C 279, including GASP/WEBT, $\it{Fermi}$ and $\it{Swift}$ data, as well as polarimetric and spectroscopic data. The X-ray and $γ$-ray light curves correlate well, with no delay > 3 hours, implying general co-spatiality of the emission regions. The $γ$-ray-optical flux-flux relation changes with activi… ▽ More We report the results of decade-long (2008-2018) $γ$-ray to 1 GHz radio monitoring of the blazar 3C 279, including GASP/WEBT, $\it{Fermi}$ and $\it{Swift}$ data, as well as polarimetric and spectroscopic data. The X-ray and $γ$-ray light curves correlate well, with no delay > 3 hours, implying general co-spatiality of the emission regions. The $γ$-ray-optical flux-flux relation changes with activity state, ranging from a linear to a more complex dependence. The behaviour of the Stokes parameters at optical and radio wavelengths, including 43 GHz VLBA images, supports either a predominantly helical magnetic field or motion of the radiating plasma along a spiral path. Apparent speeds of emission knots range from 10 to 37c, with the highest values requiring bulk Lorentz factors close to those needed to explain $γ$-ray variability on very short time scales. The Mg II emission line flux in the `blue' and `red' wings correlates with the optical synchrotron continuum flux density, possibly providing a variable source of seed photons for inverse Compton scattering. In the radio bands we find progressive delays of the most prominent light curve maxima with decreasing frequency, as expected from the frequency dependence of the $τ=1$ surface of synchrotron self-absorption. The global maximum in the 86 GHz light curve becomes less prominent at lower frequencies, while a local maximum, appearing in 2014, strengthens toward decreasing frequencies, becoming pronounced at $\sim5$ GHz. These tendencies suggest different Doppler boosting of stratified radio-emitting zones in the jet. △ Less

Submitted 17 January, 2020; originally announced January 2020.

Comments: 21 pages, 18 figures, accepted for publication in MNRAS

Journal ref: MNRAS 492 (2020) 3829

Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis

Authors: Richard J. Chen, Ming Y. Lu, **gwen Wang, Drew F. K. Williamson, Scott J. Rodig, Neal I. Lindeman, Faisal Mahmood

Abstract: Cancer diagnosis, prognosis, and therapeutic response predictions are based on morphological information from histology slides and molecular profiles from genomic data. However, most deep learning-based objective outcome prediction and grading paradigms are based on histology or genomics alone and do not make use of the complementary information in an intuitive manner. In this work, we propose Pat… ▽ More Cancer diagnosis, prognosis, and therapeutic response predictions are based on morphological information from histology slides and molecular profiles from genomic data. However, most deep learning-based objective outcome prediction and grading paradigms are based on histology or genomics alone and do not make use of the complementary information in an intuitive manner. In this work, we propose Pathomic Fusion, an interpretable strategy for end-to-end multimodal fusion of histology image and genomic (mutations, CNV, RNA-Seq) features for survival outcome prediction. Our approach models pairwise feature interactions across modalities by taking the Kronecker product of unimodal feature representations and controls the expressiveness of each representation via a gating-based attention mechanism. Following supervised learning, we are able to interpret and saliently localize features across each modality, and understand how feature importance shifts when conditioning on multimodal input. We validate our approach using glioma and clear cell renal cell carcinoma datasets from the Cancer Genome Atlas (TCGA), which contains paired whole-slide image, genotype, and transcriptome data with ground truth survival and histologic grade labels. In a 15-fold cross-validation, our results demonstrate that the proposed multimodal fusion paradigm improves prognostic determinations from ground truth grading and molecular subty**, as well as unimodal deep networks trained on histology and genomic data alone. The proposed method establishes insight and theory on how to train deep networks on multimodal biomedical data in an intuitive manner, which will be useful for other problems in medicine that seek to combine heterogeneous data streams for understanding diseases and predicting response and resistance to treatment. △ Less

Submitted 3 September, 2020; v1 submitted 18 December, 2019; originally announced December 2019.

Comments: Code and trained models are made available at: https://github.com/mahmoodlab/PathomicFusion

The Pulsar Search Collaboratory: Current Status and Future Prospects

Authors: Harsha Blumer, Maura A. McLaughlin, John Stewart, Kathryn Williamson, Duncan R. Lorimer, Sue Ann Heatherly, Joseph K. Swiggum, Ryan S. Lynch, Cabot Zabriskie, Natalia Lewandowska, Aubrey Roy, Shirley Au

Abstract: The Pulsar Search Collaboratory (PSC) is a collaboration between the Green Bank Observatory and West Virginia University, funded by the National Science Foundation. The PSC program is currently expanding nationwide and engages high school students, teachers, and undergraduate mentors in real-world research by searching for pulsars in data collected with the 100-m Green Bank Telescope. In the proce… ▽ More The Pulsar Search Collaboratory (PSC) is a collaboration between the Green Bank Observatory and West Virginia University, funded by the National Science Foundation. The PSC program is currently expanding nationwide and engages high school students, teachers, and undergraduate mentors in real-world research by searching for pulsars in data collected with the 100-m Green Bank Telescope. In the process, students learn about observational radio astronomy, radio frequency interference, pulsar timing, and data analysis procedures. The primary goals of the PSC are to stimulate student interest in Science, Technology, Engineering, and Mathematics (STEM) careers, to prepare teachers in implementing authentic research with students by training them within a professional scientific community, and to promote student use of information technologies through online activities and workshops. In this paper, we provide an overview of pulsar science and the data analysis students undertake, as well as a general overview of the program. We then discuss evaluation data collected from participants through a series of survey questions to determine if the program's initial goals were met. The program had a positive impact on the students according to multiple measures, in particular, on their understanding of the nature of scientific inquiry and motivation to pursue STEM career paths. △ Less

Submitted 11 September, 2019; originally announced September 2019.

Comments: 10 pages, 3 figures, Accepted for publication in American Journal of Physics

Embedding Climate Change Engagement in Astronomy Education and Research

Authors: Kathryn Williamson, Travis A. Rector, James Lowenthal

Abstract: This White Paper is a call to action for astronomers to respond to climate change with a large structural transition within our profession. Many astronomers are deeply concerned about climate change and act upon it in their personal and professional lives, and many organizations within astronomy have incorporated incremental changes. We need a collective impact model to better network and grow our… ▽ More This White Paper is a call to action for astronomers to respond to climate change with a large structural transition within our profession. Many astronomers are deeply concerned about climate change and act upon it in their personal and professional lives, and many organizations within astronomy have incorporated incremental changes. We need a collective impact model to better network and grow our efforts so that we can achieve results that are on the scale appropriate to address climate change at the necessary level indicated by scientific research; e.g., becoming carbon neutral by 2050. We need to implement strategies within two primary drivers of our field: (1) Education and Outreach, and (2) Research Practices and Infrastructure. (1) In the classroom and through public talks, astronomers reach a large audience. Astronomy is closely connected to the science of climate change, and it is arguably the most important topic we include in our curriculum. Due to misinformation and disinformation, climate change communication is different than for other areas of science. We therefore need to expand our communication and implement effective strategies, for which there is now a considerable body of research. (2) On a per-person basis astronomers have an outsized carbon impact. There are numerous ways we can reduce our footprint; e.g., in the design and operation of telescope facilities and in the optimization and reduction of travel. Fortunately, many of these solutions are win-win scenarios, e.g., increasing the online presence of conferences will reduce the carbon footprint while increasing participation, especially for astronomers working with fewer financial resources. Astronomers have an obligation to act on climate change in every way possible, and we need to do it now. In this White Paper, we outline a plan for collective impact using a Networked Improvement Community (NIC) approach. △ Less

Submitted 18 July, 2019; originally announced July 2019.

Comments: Submitted as a State of the Profession White Paper for the Astro2020 Decadal Survey (10 pages, 1 figure)

NANOGrav Education and Outreach: Growing a Diverse and Inclusive Collaboration for Low-Frequency Gravitational Wave Astronomy

Authors: The NANOGrav Collaboration, P. T. Baker, H. Blumer, A. Brazier, S. Chatterjee, B. Christy, F. Crawford, M. E. DeCesar, T. Dolch, N. E. Garver-Daniels, J. S. Hazboun, K. Holley-Bockelmann, D. L. Kaplan, J . S. Key, T. C. Klein, M. T. Lam, N. Lewandowska, D. R. Lorimer, R. S. Lynch, M. A. McLaughlin, N. McMann, J. Page, N. T. Palliyaguru, J. D. Romano, X. Siemens , et al. (3 additional authors not shown)

Abstract: The new field of gravitational wave astrophysics requires a growing pool of students and researchers with unique, interdisciplinary skill sets. It also offers an opportunity to build a diverse, inclusive astronomy community from the ground up. We describe the efforts used by the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) NSF Physics Frontiers Center to foster such grow… ▽ More The new field of gravitational wave astrophysics requires a growing pool of students and researchers with unique, interdisciplinary skill sets. It also offers an opportunity to build a diverse, inclusive astronomy community from the ground up. We describe the efforts used by the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) NSF Physics Frontiers Center to foster such growth by involving students at all levels in low-frequency gravitational wave astrophysics with pulsar timing arrays (PTAs) and establishing collaboration policies that ensure broad participation by diverse groups. We describe and illustrate the impact of these techniques on our collaboration as a case study for other distributed collaborations. △ Less

Submitted 17 July, 2019; originally announced July 2019.

Comments: Submitted as a State of the Profession White Paper for the Astro2020 Decadal Survey (10 pages + cover and references, 6 figures)

Touching the stars: improving NASA 3D printed data sets with blind and visually impaired audiences

Authors: Kimberly Kowal Arcand, April Jubett, Megan Watzke, Sara Price, Kelly T. S. Williamson, Peter Edmonds

Abstract: Astronomy has been an inherently visual area of science for millenia, yet a majority of its significant discoveries take place in wavelengths beyond human vision. There are many people, including those with low or no vision, who cannot participate fully in such discoveries if visual media is the primary communication mechanism. Numerous efforts have worked to address equity of accessibility to suc… ▽ More Astronomy has been an inherently visual area of science for millenia, yet a majority of its significant discoveries take place in wavelengths beyond human vision. There are many people, including those with low or no vision, who cannot participate fully in such discoveries if visual media is the primary communication mechanism. Numerous efforts have worked to address equity of accessibility to such knowledge sharing, such as through the creation of three dimensional (3D) printed data sets. This paper describes progress made through technological and programmatic developments in tactile 3D models using the NASA Chandra X-ray Observatory to improve access to data. △ Less

Submitted 14 June, 2019; originally announced June 2019.

Comments: 22 pages, 6 figures

Time dependent spectral modeling of Markarian 421 during a violent outburst in 2010

Authors: B. Banerjee, M. Joshi, P. Majumdar, K. E. Williamson, S. G. Jorstad, A. P. Marscher

Abstract: We present the results of extensive modeling of the spectral energy distributions (SEDs) of the closest blazar (z=0.031) Markarian 421 (Mrk 421) during a giant outburst in February 2010. The source underwent rapid flux variations in both X-rays and very high energy (VHE) gamma-rays as it evolved from a low-flux state on 2010 February 13-15 to a high-flux state on 2010 February 17. During this peri… ▽ More We present the results of extensive modeling of the spectral energy distributions (SEDs) of the closest blazar (z=0.031) Markarian 421 (Mrk 421) during a giant outburst in February 2010. The source underwent rapid flux variations in both X-rays and very high energy (VHE) gamma-rays as it evolved from a low-flux state on 2010 February 13-15 to a high-flux state on 2010 February 17. During this period, the source exhibited significant spectral hardening from X-rays to VHE gamma-rays while exhibiting a "harder when brighter" behavior in these energy bands. We reproduce the broadband SED using a time-dependent multi-zone leptonic jet model with radiation feedback. We find that an injection of the leptonic particle population with a single power-law energy distribution at shock fronts followed by energy losses in an inhomogeneous emission region is suitable for explaining the evolution of Mrk 421 from low- to high-flux state in February 2010. The spectral states are successfully reproduced by a combination of a few key physical parameters, such as the maximum $\&$ minimum cutoffs and power-law slope of the electron injection energies, magnetic field strength, and bulk Lorentz factor of the emission region. The simulated light curves and spectral evolution of Mrk 421 during this period imply an almost linear correlation between X-ray flux at 1-10 keV energies and VHE gamma-ray flux above 200 GeV, as has been previously exhibited by this source. Through this study, a general trend that has emerged for the role of physical parameters is that, as the flare evolves from a low- to a high-flux state, higher bulk kinetic energy is injected into the system with a harder particle population and a lower magnetic field strength. △ Less

Submitted 3 May, 2019; originally announced May 2019.

Comments: 13 pages, 5 figures, accepted for publication in MNRAS

A flat persistence diagram for improved visualization of persistent homology

Authors: Raoul R. Wadhwa, Andrew Dhawan, Drew F. K. Williamson, Jacob G. Scott

Abstract: Visualization in the emerging field of topological data analysis has progressed from persistence barcodes and persistence diagrams to display of two-parameter persistent homology. Although persistence barcodes and diagrams have permitted insight into the geometry underlying complex datasets, visualization of even single-parameter persistent homology has significant room for improvement. Here, we p… ▽ More Visualization in the emerging field of topological data analysis has progressed from persistence barcodes and persistence diagrams to display of two-parameter persistent homology. Although persistence barcodes and diagrams have permitted insight into the geometry underlying complex datasets, visualization of even single-parameter persistent homology has significant room for improvement. Here, we propose a modification to the conventional persistence diagram - the flat persistence diagram - that more efficiently displays information relevant to persistent homology and simultaneously corrects for visual bias present in the former. Flat persistence diagrams display equivalent information as their predecessor, while providing researchers with an intuitive horizontal reference axis in contrast to the usual diagonal reference line. Reducing visual bias through the use of appropriate graphical displays not only provides more accurate, but also deeper insights into the topology that underlies complex datasets. Introducing flat persistence diagrams into widespread use would bring researchers one step closer to practical application of topological data analysis. △ Less

Submitted 5 January, 2019; v1 submitted 11 December, 2018; originally announced December 2018.

Comments: 4 pages, 2 figures

The magnetic field structure in CTA 102 from high resolution mm-VLBI observations during the flaring state in 2016-2017

Authors: Carolina Casadio, Alan P. Marscher, Svetlana G. Jorstad, Dmitry A. Blinov, Nicholas R. MacDonald, Thomas P. Krichbaum, Biagina Boccardi, Efthalia Traianou, José L. Gómez, Iván Agudo, Bong-Won Sohn, Michael Bremer, Jeffrey Hodgson, Juha Kallunki, Jae-Young Kim, Karen E. Williamson, J. Anton Zensus

Abstract: Investigating the magnetic field structure in the innermost regions of relativistic jets is fundamental to shed light on the crucial physical processes giving rise to the jet formation, as well as to its extraordinary radiation output up to gamma-ray energies. We study the magnetic field structure of the quasar CTA 102 with 3 and 7 mm-VLBI polarimetric observations, reaching an unprecedented resol… ▽ More Investigating the magnetic field structure in the innermost regions of relativistic jets is fundamental to shed light on the crucial physical processes giving rise to the jet formation, as well as to its extraordinary radiation output up to gamma-ray energies. We study the magnetic field structure of the quasar CTA 102 with 3 and 7 mm-VLBI polarimetric observations, reaching an unprecedented resolution (~50 microarcsec). We also investigate the variability and physical process occurring in the source during the observing period which coincides with a very active state of the source till high-energies. The Faraday rotation analysis between 3 and 7mm shows a gradient in rotation measure with a maximum value of ~6X10^4 rad/m^2 and intrinsic electric vector position angles (EVPAs) oriented around the centroid of the core, suggesting the presence of large-scale helical magnetic fields. Such a magnetic field structure is also visible in 7 mm images when a new superluminal component is crossing the core region. The 7mm EVPAs orientation is different when the component is exiting the core or crossing a stationary feature at ~0.1 mas. The interaction between the superluminal component and a recollimation shock at ~0.1 mas could have triggered the multi-wavelengths flares. The variability Doppler factor associated with such interaction is large enough to explain the high energy emission, as we infer from the analysis of gamma-ray and X-ray data, and it is in agreement with the Doppler factor obtained to explain the extraordinary optical flare by Raiteri et al.(2017). △ Less

Submitted 19 February, 2019; v1 submitted 5 December, 2018; originally announced December 2018.

Comments: Published in Astronomy & Astrophysics

Journal ref: A&A 622, A158 (2019)

A posteriori error estimates and adaptive mesh refinement for the Stokes-Brinkman problem

Authors: Kevin Williamson, Pavel Burda, Bedřich Sousedík

Abstract: The Stokes-Brinkman equations model flow in heterogeneous porous media by combining the Stokes and Darcy models of flow into a single system of equations. With suitable parameters, the equations can model either flow without detailed knowledge of the interface between the two regions. Thus, the Stokes-Brinkman equations provide an alternative to coupled Darcy-Stokes models. After a brief review of… ▽ More The Stokes-Brinkman equations model flow in heterogeneous porous media by combining the Stokes and Darcy models of flow into a single system of equations. With suitable parameters, the equations can model either flow without detailed knowledge of the interface between the two regions. Thus, the Stokes-Brinkman equations provide an alternative to coupled Darcy-Stokes models. After a brief review of the Stokes-Brinkman problem and its discretization using Taylor-Hood finite elements, we present a residual-based a posteriori error estimate and use it to drive an adaptive mesh refinement process. We compare several strategies for the mesh refinement, and demonstrate its effectiveness by numerical experiments in both 2D and 3D. △ Less

Submitted 31 May, 2019; v1 submitted 27 September, 2018; originally announced September 2018.

Comments: 20 pages, 15 figures

MSC Class: 65N30; 65N12

Journal ref: Mathematics and Computers in Simulation, 166, 266-282, 2019

X-ray, UV, and Radio Timing Observations of the Radio Galaxy 3C 120

Authors: Alan P. Marscher, Svetlana G. Jorstad, Karen E. Williamson, Anne Lähteenmäki, Merja Tornikoski, John M. Hunter, Katya A. Leidig, Muhammad Zain Mobeen, Rafael J. C. Vera, Wara Chamani

Abstract: We report the results of monitoring of the radio galaxy 3C 120 with the Neil Gehrels Swift Observatory, Very Long Baseline Array, and Metsähovi Radio Observatory. The UV-optical continuum spectrum and R-band polarization can be explained by a superposition of an inverted-spectrum source with a synchrotron component containing a disordered magnetic field. The UV-optical and X-ray light curves inclu… ▽ More We report the results of monitoring of the radio galaxy 3C 120 with the Neil Gehrels Swift Observatory, Very Long Baseline Array, and Metsähovi Radio Observatory. The UV-optical continuum spectrum and R-band polarization can be explained by a superposition of an inverted-spectrum source with a synchrotron component containing a disordered magnetic field. The UV-optical and X-ray light curves include dips and flares, while several superluminal knots appear in the parsec-scale jet. The recovery time of the second dip was longer at UV-optical wavelengths, in conflict with a model in which the inner accretion disk (AD) is disrupted during a dip and then refilled from outer to inner radii. We favor an alternative scenario in which occasional polar alignments of the magnetic field in the disk and corona cause the flux dips and formation of shocks in the jet. Similar to observations of Seyfert galaxies, intra-band time lags of flux variations are longer than predicted by the standard AD model. This suggests that scattering or some other reprocessing occurs. The 37 GHz light curve is well correlated with the optical-UV variations, with a ~20-day delay. A radio flare in the jet occurred in a superluminal knot 0.14 milliarcseconds downstream of the 43 GHz "core," which places the site of the preceding X-ray/UV/optical flare within the core 0.5-1.3 pc from the black hole. The inverted UV-optical flare spectrum can be explained by a nearly mono-energetic electron distribution with energy similar to the minimum energy inferred in the TeV gamma-ray emitting regions of some BL Lacertae objects. △ Less

Submitted 25 September, 2018; originally announced September 2018.

Comments: Accepted for publication in the Astrophysical Journal

The Pulsar Search Collaboratory: Expanding Nationwide

Authors: Kathryn Williamson, Maura McLaughlin, Sue Ann Heatherly, John Stewart, Duncan Lorimer, Harsha Blumer, Cabot Zabriskie, Ryan Lynch

Abstract: The Pulsar Search Collaboratory (PSC) engages high school students and teachers in analyzing real data from the Robert C. Byrd Green Bank Telescope for the purpose of discovering exotic stars called pulsars. These cosmic clocks can be used as a galactic-scale detector of gravitational waves, ripples in space-time that have recently been directly detected from the mergers of stellar-mass black hole… ▽ More The Pulsar Search Collaboratory (PSC) engages high school students and teachers in analyzing real data from the Robert C. Byrd Green Bank Telescope for the purpose of discovering exotic stars called pulsars. These cosmic clocks can be used as a galactic-scale detector of gravitational waves, ripples in space-time that have recently been directly detected from the mergers of stellar-mass black holes. Through immersing students in an authentic, positive learning environment to build a sense of belonging and competency, the goal of the PSC is to promote students' long-term interests in science and science careers. PSC students have discovered 7 pulsars since the start of the PSC in 2008. Originally targeted at teachers and students in West Virginia, over time the program has grown to 18 states. In a new effort to scale the PSC nationally, the PSC has developed an integrated online training program with both self-guided lectures and homework and real-time interactions with pulsar astronomers. Now, any high school student can join in the exciting search for pulsars and the discovery of a new type of gravitational waves. △ Less

Submitted 16 July, 2018; originally announced July 2018.

Comments: 6 pages, 1 figure, accepted for publication in The Physics Teacher

A multi-wavelength polarimetric study of the blazar CTA 102 during a Gamma-ray flare in 2012

Authors: Carolina Casadio, José L. Gómez, Svetlana G. Jorstad, Alan P. Marscher, Valeri M. Larionov, Paul S. Smith, Mark A. Gurwell, Anne Lähteenmäki, Iván Agudo, Sol N. Molina, Vishal Bala, Manasvita Joshi, Brian Taylor, Karen E. Williamson, Arkady A. Arkharov, Dmitry A. Blinov, George A. Borman, Andrea Di Paola, Tatiana S. Grishina, Vladimir A. Hagen-Thorn, Ryosuke Itoh, Evgenia N. Kopatskaya, Elena G. Larionova, Liudmila V. Larionova, Daria A. Morozova , et al. (6 additional authors not shown)

Abstract: We perform a multi-wavelength polarimetric study of the quasar CTA 102 during an extraordinarily bright $γ$-ray outburst detected by the {\it Fermi} Large Area Telescope in September-October 2012 when the source reached a flux of F$_{>100~\mathrm{MeV}} =5.2\pm0.4\times10^{-6}$ photons cm$^{-2}$ s$^{-1}$. At the same time the source displayed an unprecedented optical and NIR outburst. We study the… ▽ More We perform a multi-wavelength polarimetric study of the quasar CTA 102 during an extraordinarily bright $γ$-ray outburst detected by the {\it Fermi} Large Area Telescope in September-October 2012 when the source reached a flux of F$_{>100~\mathrm{MeV}} =5.2\pm0.4\times10^{-6}$ photons cm$^{-2}$ s$^{-1}$. At the same time the source displayed an unprecedented optical and NIR outburst. We study the evolution of the parsec scale jet with ultra-high angular resolution through a sequence of 80 total and polarized intensity Very Long Baseline Array images at 43 GHz, covering the observing period from June 2007 to June 2014. We find that the $γ$-ray outburst is coincident with flares at all the other frequencies and is related to the passage of a new superluminal knot through the radio core. The powerful $γ$-ray emission is associated with a change in direction of the jet, which became oriented more closely to our line of sight ($θ\sim$1.2$^{\circ}$) during the ejection of the knot and the $γ$-ray outburst. During the flare, the optical polarized emission displays intra-day variability and a clear clockwise rotation of EVPAs, which we associate with the path followed by the knot as it moves along helical magnetic field lines, although a random walk of the EVPA caused by a turbulent magnetic field cannot be ruled out. We locate the $γ$-ray outburst a short distance downstream of the radio core, parsecs from the black hole. This suggests that synchrotron self-Compton scattering of near-infrared to ultraviolet photons is the probable mechanism for the $γ$-ray production. △ Less

Submitted 10 September, 2015; v1 submitted 28 August, 2015; originally announced August 2015.

Comments: Accepted for publication in The Astrophysical Journal

Comprehensive Monitoring of Gamma-ray Bright Blazars. I. Statistical Study of Optical, X-ray, and Gamma-ray Spectral Slopes

Authors: Karen E. Williamson, Svetlana G. Jorstad, Alan P. Marscher, Valeri M. Larionov, Paul S. Smith, Iván Agudo, Arkady A. Arkharov, Dmitry A. Blinov, Carolina Casadio, Natalia V. Efimova, José L. Gómez, Vladimir A. Hagen-Thorn, Manasvita Joshi, Tatiana S. Konstantinova, Evgenia N. Kopatskaya, Elena G. Larionova, Liudmilla V. Larionova, Michael P. Malmrose, Ian M. McHardy, Sol N. Molina, Daria A. Morozova, Gary D. Schmidt, Brian W. Taylor, Ivan S. Troitsky

Abstract: We present $γ$-ray, X-ray, ultraviolet, optical, and near-infrared light curves of 33 $γ$-ray bright blazars over four years that we have been monitoring since 2008 August with multiple optical, ground-based telescopes and the Swift satellite, and augmented by data from the Fermi Gamma-ray Space Telescope and other publicly available data from Swift. The sample consists of 21 flat-spectrum radio q… ▽ More We present $γ$-ray, X-ray, ultraviolet, optical, and near-infrared light curves of 33 $γ$-ray bright blazars over four years that we have been monitoring since 2008 August with multiple optical, ground-based telescopes and the Swift satellite, and augmented by data from the Fermi Gamma-ray Space Telescope and other publicly available data from Swift. The sample consists of 21 flat-spectrum radio quasars (FSRQs) and 12 BL Lac objects (BL Lacs). We identify quiescent and active states of the sources based on their $γ$-ray behavior. We derive $γ$-ray, X-ray, and optical spectral indices, $α_γ$, $α_X$, and $α_o$, respectively ($F_ν\proptoν^α$), and construct spectral energy distributions (SEDs) during quiescent and active states. We analyze the relationships between different spectral indices, blazar classes, and activity states. We find (i) significantly steeper $γ$-ray spectra of FSRQs than for BL Lacs during quiescent states, but a flattening of the spectra for FSRQs during active states while the BL Lacs show no significant change; (ii) a small difference of $α_X$ within each class between states, with BL Lac X-ray spectra significantly steeper than in FSRQs; (iii) a highly peaked distribution of X-ray spectral slopes of FSRQs at $\sim-$0.60, but a very broad distribution of $α_X$ of BL Lacs during active states; (iv) flattening of the optical spectra of FSRQs during quiescent states, but no statistically significant change of $α_o$ of BL Lacs between states; and (v) a positive correlation between optical and $γ$-ray spectral slopes of BL Lacs, with similar values of the slopes. We discuss the findings with respect to the relative prominence of different components of high-energy and optical emission as the flux state changes. △ Less

Submitted 10 June, 2014; originally announced June 2014.

Comments: 40 pages, 28 figures, 10 Tables. Additional figures at http://www.bu.edu/blazars/VLBAproject.html. Accepted by ApJ

A Tight Connection between Gamma-Ray Outbursts and Parsec-Scale Jet Activity in the Quasar 3C 454.3

Authors: Svetlana G. Jorstad, Alan P. Marscher, Paul S. Smith, Valeri M. Larionov, Iván Agudo, Mark Gurwell, Ann E. Wehrle, Anne Lähteenmäki, Maria G. Nikolashvili, Gary D. Schmidt, Arkady A. Arkharov, Dmitry A. Blinov, Kelly Blumenthal, Carolina Casadio, Revaz A. Chigladze, Natatia V. Efimova, Joseph R. Eggen, José L. Gómez, Dirk Grupe, Vladimir A. Hagen-Thorn, Manasvita Joshi, Givi N. Kimeridze, Tatiana S. Konstantinova, Evgenia N. Kopatskaya, Omar M. Kurtanidze , et al. (20 additional authors not shown)

Abstract: We analyze the multifrequency behavior of the quasar 3C 454.3 during three prominent γ-ray outbursts: 2009 Autumn, 2010 Spring, and 2010 Autumn. The data reveal a repeating pattern, including a triple flare structure, in the properties of each γ-ray outburst, which implies similar mechanism(s) and location for all three events. The multi-frequency behavior indicates that the lower frequency events… ▽ More We analyze the multifrequency behavior of the quasar 3C 454.3 during three prominent γ-ray outbursts: 2009 Autumn, 2010 Spring, and 2010 Autumn. The data reveal a repeating pattern, including a triple flare structure, in the properties of each γ-ray outburst, which implies similar mechanism(s) and location for all three events. The multi-frequency behavior indicates that the lower frequency events are co-spatial with the γ-ray outbursts, although the γ-ray emission varies on the shortest timescales. We determine that the variability from UV to IR wavelengths during an outburst results from a single synchrotron component whose properties do not change significantly over the different outbursts. Despite a general increase in the degree of optical linear polarization during an outburst, the polarization drops significantly at the peak of the γ-ray event, which suggests that both shocks and turbulent processes are involved. We detect two disturbances (knots) with superluminal apparent speeds in the parsec-scale jet associated with the outbursts in 2009 Autumn and 2010 Autumn. The kinematic properties of the knots can explain the difference in amplitudes of the γ-ray events, while their millimeter-wave polarization is related to the optical polarization during the outbursts. We interpret the multi-frequency behavior within models involving either a system of standing conical shocks or magnetic reconnection events located in the parsec-scale millimeter-wave core of the jet. We argue that γ-ray outbursts with variability timescales as short as ~ 3 hr can occur on parsec scales if flares take place in localized regions such as turbulent cells. △ Less

Submitted 9 July, 2013; originally announced July 2013.

Comments: 80 pages, 10 Tables, 31 figures, accepted for publication in ApJ

Multiwavelength Variations of 3C 454.3 during the November 2010 to January 2011 Outburst

Authors: Ann E. Wehrle, Alan P. Marscher, Svetlana G. Jorstad, Mark A. Gurwell, Manasvita Joshi, Nicholas R. MacDonald, Karen E. Williamson, Ivan Agudo, Dirk Grupe

Abstract: We present multiwavelength data of the blazar 3C 454.3 obtained during an extremely bright outburst from November 2010 through January 2011. These include flux density measurements with the Herschel Space Observatory at five submillimeter-wave and far-infrared bands, the Fermi Large Area Telescope at gamma-ray energies, Swift at X-ray, ultraviolet (UV), and optical frequencies, and the Submillimet… ▽ More We present multiwavelength data of the blazar 3C 454.3 obtained during an extremely bright outburst from November 2010 through January 2011. These include flux density measurements with the Herschel Space Observatory at five submillimeter-wave and far-infrared bands, the Fermi Large Area Telescope at gamma-ray energies, Swift at X-ray, ultraviolet (UV), and optical frequencies, and the Submillimeter Array at 1.3 mm. From this dataset, we form a series of 52 spectral energy distributions (SEDs) spanning nearly two months that are unprecedented in time coverage and breadth of frequency. Discrete correlation anlaysis of the millimeter, far-infrared, and gamma-ray light curves show that the variations were essentially simultaneous, indicative of co-spatiality of the emission, at these wavebands. In contrast, differences in short-term fluctuations at various wavelengths imply the presence of inhomegeneities in physical conditions across the source. We locate the site of the outburst in the parsec-scale core, whose flux density as measured on 7 mm Very Long Baseline Array images increased by 70 percent during the first five weeks of the outburst. Based on these considerations and guided by the SEDs, we propose a model in which turbulent plasma crosses a conical standing shock in the parsec-scale region of the jet. Here, the high-energy emission in the model is produced by inverse Compton scattering of seed photons supplied by either nonthermal radiation from a Mach disk, thermal emission from hot dust, or (for X-rays) synchrotron radiation from plasma that crosses the standing shock. For the two dates on which we fitted the model SED to the data, the model corresponds very well to the observations at all bands except at X-ray energies, where the spectrum is flatter than observed. △ Less

Submitted 22 August, 2012; originally announced August 2012.

Comments: Accepted for publication in Astrophysical Journal. 82 pages, 13 figures

Parsec-Scale Jet Behavior of the Quasar 3C 454.3 during the High Gamma-Ray States in 2009 and 2010

Authors: S. G. Jorstad, A. P. Marscher, M. Joshi, N. R. MacDonald, T. L. Scott, K. E. Williamson, P. S. Smith, V. M. Larionov, I. Agudo, M. Gurwell

Abstract: We analyze total and polarized intensity images of the quasar 3C 454.3 obtained monthly with the VLBA at 43 GHz within the ongoing Boston U. monitoring program of gamma-ray blazars started in June 2007. The data are supplemented by VLBA observations performed during intense campaigns of 2 week duration when the quasar was observed 3 times per campaign. We find a strong increase of activity in the… ▽ More We analyze total and polarized intensity images of the quasar 3C 454.3 obtained monthly with the VLBA at 43 GHz within the ongoing Boston U. monitoring program of gamma-ray blazars started in June 2007. The data are supplemented by VLBA observations performed during intense campaigns of 2 week duration when the quasar was observed 3 times per campaign. We find a strong increase of activity in the parsec-scale jet of the quasar during high gamma-ray states in December 2009, April 2010, and November 2010. We detect new superluminal knots, K09 and K10, associated with the autumn 2009 and 2010 outbursts, respectively, and compare their kinematic parameters. We analyze optical polarimetric behavior along with polarization parameters of the parsec-scale jet and outline similarities and differences in polarization properties across wavelengths. The results of the analysis support the conclusions that the optical polarized emission is produced in a region located in the vicinity of the mm-wave core of the jet of the quasar, and that the gamma-ray outbursts occur when a superluminal disturbance passes through the core. △ Less

Submitted 2 May, 2012; originally announced May 2012.

Comments: 2012 Fermi & Jansky Proceedings - eConf C1111101