-
Autonomous Quality and Hallucination Assessment for Virtual Tissue Staining and Digital Pathology
Authors:
Luzhe Huang,
Yuzhu Li,
Nir Pillar,
Tal Keidar Haran,
William Dean Wallace,
Aydogan Ozcan
Abstract:
Histopathological staining of human tissue is essential in the diagnosis of various diseases. The recent advances in virtual tissue staining technologies using AI alleviate some of the costly and tedious steps involved in the traditional histochemical staining process, permitting multiplexed rapid staining of label-free tissue without using staining reagents, while also preserving tissue. However,…
▽ More
Histopathological staining of human tissue is essential in the diagnosis of various diseases. The recent advances in virtual tissue staining technologies using AI alleviate some of the costly and tedious steps involved in the traditional histochemical staining process, permitting multiplexed rapid staining of label-free tissue without using staining reagents, while also preserving tissue. However, potential hallucinations and artifacts in these virtually stained tissue images pose concerns, especially for the clinical utility of these approaches. Quality assessment of histology images is generally performed by human experts, which can be subjective and depends on the training level of the expert. Here, we present an autonomous quality and hallucination assessment method (termed AQuA), mainly designed for virtual tissue staining, while also being applicable to histochemical staining. AQuA achieves 99.8% accuracy when detecting acceptable and unacceptable virtually stained tissue images without access to ground truth, also presenting an agreement of 98.5% with the manual assessments made by board-certified pathologists. Besides, AQuA achieves super-human performance in identifying realistic-looking, virtually stained hallucinatory images that would normally mislead human diagnosticians by deceiving them into diagnosing patients that never existed. We further demonstrate the wide adaptability of AQuA across various virtually and histochemically stained tissue images and showcase its strong external generalization to detect unseen hallucination patterns of virtual staining network models as well as artifacts observed in the traditional histochemical staining workflow. This framework creates new opportunities to enhance the reliability of virtual staining and will provide quality assurance for various image generation and transformation tasks in digital pathology and computational imaging.
△ Less
Submitted 29 April, 2024;
originally announced April 2024.
-
The Ethics of Advanced AI Assistants
Authors:
Iason Gabriel,
Arianna Manzini,
Geoff Keeling,
Lisa Anne Hendricks,
Verena Rieser,
Hasan Iqbal,
Nenad TomaĊĦev,
Ira Ktena,
Zachary Kenton,
Mikel Rodriguez,
Seliem El-Sayed,
Sasha Brown,
Canfer Akbulut,
Andrew Trask,
Edward Hughes,
A. Stevie Bergman,
Renee Shelby,
Nahema Marchal,
Conor Griffin,
Juan Mateos-Garcia,
Laura Weidinger,
Winnie Street,
Benjamin Lange,
Alex Ingerman,
Alison Lentz
, et al. (32 additional authors not shown)
Abstract:
This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, pro…
▽ More
This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, providing an overview of AI assistants, their technical foundations and potential range of applications. It then explores questions around AI value alignment, well-being, safety and malicious uses. Extending the circle of inquiry further, we next consider the relationship between advanced AI assistants and individual users in more detail, exploring topics such as manipulation and persuasion, anthropomorphism, appropriate relationships, trust and privacy. With this analysis in place, we consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants. Finally, we conclude by providing a range of recommendations for researchers, developers, policymakers and public stakeholders.
△ Less
Submitted 28 April, 2024; v1 submitted 24 April, 2024;
originally announced April 2024.
-
Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning
Authors:
Xilin Yang,
Bijie Bai,
Yijie Zhang,
Musa Aydin,
Sahan Yoruc Selcuk,
Zhen Guo,
Gregory A. Fishbein,
Karine Atlan,
William Dean Wallace,
Nir Pillar,
Aydogan Ozcan
Abstract:
Systemic amyloidosis is a group of diseases characterized by the deposition of misfolded proteins in various organs and tissues, leading to progressive organ dysfunction and failure. Congo red stain is the gold standard chemical stain for the visualization of amyloid deposits in tissue sections, as it forms complexes with the misfolded proteins and shows a birefringence pattern under polarized lig…
▽ More
Systemic amyloidosis is a group of diseases characterized by the deposition of misfolded proteins in various organs and tissues, leading to progressive organ dysfunction and failure. Congo red stain is the gold standard chemical stain for the visualization of amyloid deposits in tissue sections, as it forms complexes with the misfolded proteins and shows a birefringence pattern under polarized light microscopy. However, Congo red staining is tedious and costly to perform, and prone to false diagnoses due to variations in the amount of amyloid, staining quality and expert interpretation through manual examination of tissue under a polarization microscope. Here, we report the first demonstration of virtual birefringence imaging and virtual Congo red staining of label-free human tissue to show that a single trained neural network can rapidly transform autofluorescence images of label-free tissue sections into brightfield and polarized light microscopy equivalent images, matching the histochemically stained versions of the same samples. We demonstrate the efficacy of our method with blind testing and pathologist evaluations on cardiac tissue where the virtually stained images agreed well with the histochemically stained ground truth images. Our virtually stained polarization and brightfield images highlight amyloid birefringence patterns in a consistent, reproducible manner while mitigating diagnostic challenges due to variations in the quality of chemical staining and manual imaging processes as part of the clinical workflow.
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
Virtual histological staining of unlabeled autopsy tissue
Authors:
Yuzhu Li,
Nir Pillar,
**gxi Li,
Tairan Liu,
Di Wu,
Songyu Sun,
Guangdong Ma,
Kevin de Haan,
Luzhe Huang,
Sepehr Hamidi,
Anatoly Urisman,
Tal Keidar Haran,
William Dean Wallace,
Jonathan E. Zuckerman,
Aydogan Ozcan
Abstract:
Histological examination is a crucial step in an autopsy; however, the traditional histochemical staining of post-mortem samples faces multiple challenges, including the inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, as well as the resource-intensive nature of chemical staining procedures covering large tissue areas, which demand substantial labor, cost, a…
▽ More
Histological examination is a crucial step in an autopsy; however, the traditional histochemical staining of post-mortem samples faces multiple challenges, including the inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, as well as the resource-intensive nature of chemical staining procedures covering large tissue areas, which demand substantial labor, cost, and time. These challenges can become more pronounced during global health crises when the availability of histopathology services is limited, resulting in further delays in tissue fixation and more severe staining artifacts. Here, we report the first demonstration of virtual staining of autopsy tissue and show that a trained neural network can rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images that match hematoxylin and eosin (H&E) stained versions of the same samples, eliminating autolysis-induced severe staining artifacts inherent in traditional histochemical staining of autopsied tissue. Our virtual H&E model was trained using >0.7 TB of image data and a data-efficient collaboration scheme that integrates the virtual staining network with an image registration network. The trained model effectively accentuated nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining failed to provide consistent staining quality. This virtual autopsy staining technique can also be extended to necrotic tissue, and can rapidly and cost-effectively generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
-
One-Dimensional Deep Image Prior for Curve Fitting of S-Parameters from Electromagnetic Solvers
Authors:
Sriram Ravula,
Varun Gorti,
Bo Deng,
Swagato Chakraborty,
James **enot,
Bhyrav Mutnury,
Doug Wallace,
Doug Winterberg,
Adam Klivans,
Alexandros G. Dimakis
Abstract:
A key problem when modeling signal integrity for passive filters and interconnects in IC packages is the need for multiple S-parameter measurements within a desired frequency band to obtain adequate resolution. These samples are often computationally expensive to obtain using electromagnetic (EM) field solvers. Therefore, a common approach is to select a small subset of the necessary samples and u…
▽ More
A key problem when modeling signal integrity for passive filters and interconnects in IC packages is the need for multiple S-parameter measurements within a desired frequency band to obtain adequate resolution. These samples are often computationally expensive to obtain using electromagnetic (EM) field solvers. Therefore, a common approach is to select a small subset of the necessary samples and use an appropriate fitting mechanism to recreate a densely-sampled broadband representation. We present the first deep generative model-based approach to fit S-parameters from EM solvers using one-dimensional Deep Image Prior (DIP). DIP is a technique that optimizes the weights of a randomly-initialized convolutional neural network to fit a signal from noisy or under-determined measurements. We design a custom architecture and propose a novel regularization inspired by smoothing splines that penalizes discontinuous jumps. We experimentally compare DIP to publicly available and proprietary industrial implementations of Vector Fitting (VF), the industry-standard tool for fitting S-parameters. Relative to publicly available implementations of VF, our method shows superior performance on nearly all test examples using only 5-15% of the frequency samples. Our method is also competitive to proprietary VF tools and often outperforms them for challenging input instances.
△ Less
Submitted 6 June, 2023;
originally announced June 2023.
-
Using Elo Rating as a Metric for Comparative Judgement in Educational Assessment
Authors:
Andy Gray,
Alma Rahat,
Tom Crick,
Stephen Lindsay,
Darren Wallace
Abstract:
Marking and feedback are essential features of teaching and learning, across the overwhelming majority of educational settings and contexts. However, it can take a great deal of time and effort for teachers to mark assessments, and to provide useful feedback to the students. Furthermore, it also creates a significant cognitive load on the assessors, especially in ensuring fairness and equity. Ther…
▽ More
Marking and feedback are essential features of teaching and learning, across the overwhelming majority of educational settings and contexts. However, it can take a great deal of time and effort for teachers to mark assessments, and to provide useful feedback to the students. Furthermore, it also creates a significant cognitive load on the assessors, especially in ensuring fairness and equity. Therefore, an alternative approach to marking called comparative judgement (CJ) has been proposed in the educational space. Inspired by the law of comparative judgment (LCJ). This pairwise comparison for as many pairs as possible can then be used to rank all submissions. Studies suggest that CJ is highly reliable and accurate while making it quick for the teachers. Alternative studies have questioned this claim suggesting that the process can increase bias in the results as the same submission is shown many times to an assessor for increasing reliability. Additionally, studies have also found that CJ can result in the overall marking process taking longer than a more traditional method of marking as information about many pairs must be collected.
In this paper, we investigate Elo, which has been extensively used in rating players in zero-sum games such as chess. We experimented on a large-scale Twitter dataset on the topic of a recent major UK political event ("Brexit", the UK's political exit from the European Union) to ask users which tweet they found funnier between a pair selected from ten tweets. Our analysis of the data reveals that the Elo rating is statistically significantly similar to the CJ ranking with a Kendall's tau score of 0.96 and a p-value of 1.5x10^(-5). We finish with an informed discussion regarding the potential wider application of this approach to a range of educational contexts.
△ Less
Submitted 4 April, 2022;
originally announced April 2022.
-
Experimental Evaluation of a Hierarchical Operating Framework for Ground Robots in Agriculture
Authors:
Stuart Eiffert,
Nathan D. Wallace,
He Kong,
Navid Pirmarzdashti,
Salah Sukkarieh
Abstract:
For mobile robots to be effectively applied to real world unstructured environments -- such as large scale farming -- they require the ability to generate adaptive plans that account both for limited onboard resources, and the presence of dynamic changes, including nearby moving individuals. This work provides a real world empirical evaluation of our proposed hierarchical framework for long-term a…
▽ More
For mobile robots to be effectively applied to real world unstructured environments -- such as large scale farming -- they require the ability to generate adaptive plans that account both for limited onboard resources, and the presence of dynamic changes, including nearby moving individuals. This work provides a real world empirical evaluation of our proposed hierarchical framework for long-term autonomy of field robots, conducted on University of Sydney's Swagbot agricultural robot platform. We demonstrate the ability of the framework to navigate an unstructured and dynamic environment in an effective manner, validating its use for long-term deployment in large scale farming, for tasks such as autonomous weeding in the presence of moving individuals.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
Resource and Response Aware Path Planning for Long-term Autonomy of Ground Robots in Agriculture
Authors:
Stuart Eiffert,
Nathan D. Wallace,
He Kong,
Navid Pirmarzdashti,
Salah Sukkarieh
Abstract:
Achieving long-term autonomy for mobile robots operating in real-world unstructured environments such as farms remains a significant challenge. This is made increasingly complex in the presence of moving humans or livestock. These environments require a robot to be adaptive in its immediate plans, accounting for the state of nearby individuals and the response that they might have to the robot's a…
▽ More
Achieving long-term autonomy for mobile robots operating in real-world unstructured environments such as farms remains a significant challenge. This is made increasingly complex in the presence of moving humans or livestock. These environments require a robot to be adaptive in its immediate plans, accounting for the state of nearby individuals and the response that they might have to the robot's actions. Additionally, in order to achieve longer-term goals, consideration of the limited on-board resources available to the robot is required, especially for extended missions such as weeding an agricultural field. To achieve efficient long-term autonomy, it is thus crucial to understand the impact that online dynamic updates to an energy efficient offline plan might have on resource usage whilst navigating through crowds or herds. To address these challenges, a hierarchical planning framework is proposed, integrating an online local dynamic path planner with an offline longer-term objective-based planner. This framework acts to achieve long-term autonomy through awareness of both dynamic responses of individuals to a robot's motion and the limited resources available. This paper details the hierarchical approach and its integration on a robotic platform, including a comprehensive description of the planning framework and associated perception modules. The approach is evaluated in real-world trials on farms, requiring both consideration of limited battery capacity and the presence of nearby moving individuals. These trials additionally demonstrate the ability of the framework to adapt resource use through variation of the local dynamic planner, allowing adaptive behaviour in changing environments. A summary video is available at https://youtu.be/DGVTrYwJ304.
△ Less
Submitted 22 May, 2021;
originally announced May 2021.
-
A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning
Authors:
Jiayun Li,
Wenyuan Li,
Anthony Sisk,
Huihui Ye,
W. Dean Wallace,
William Speier,
Corey W. Arnold
Abstract:
Histopathological images provide rich information for disease diagnosis. Large numbers of histopathological images have been digitized into high resolution whole slide images, opening opportunities in develo** computational image analysis tools to reduce pathologists' workload and potentially improve inter- and intra- observer agreement. Most previous work on whole slide image analysis has focus…
▽ More
Histopathological images provide rich information for disease diagnosis. Large numbers of histopathological images have been digitized into high resolution whole slide images, opening opportunities in develo** computational image analysis tools to reduce pathologists' workload and potentially improve inter- and intra- observer agreement. Most previous work on whole slide image analysis has focused on classification or segmentation of small pre-selected regions-of-interest, which requires fine-grained annotation and is non-trivial to extend for large-scale whole slide analysis. In this paper, we proposed a multi-resolution multiple instance learning model that leverages saliency maps to detect suspicious regions for fine-grained grade prediction. Instead of relying on expensive region- or pixel-level annotations, our model can be trained end-to-end with only slide-level labels. The model is developed on a large-scale prostate biopsy dataset containing 20,229 slides from 830 patients. The model achieved 92.7% accuracy, 81.8% Cohen's Kappa for benign, low grade (i.e. Grade group 1) and high grade (i.e. Grade group >= 2) prediction, an area under the receiver operating characteristic curve (AUROC) of 98.2% and an average precision (AP) of 97.4% for differentiating malignant and benign slides. The model obtained an AUROC of 99.4% and an AP of 99.8% for cancer detection on an external dataset.
△ Less
Submitted 5 November, 2020;
originally announced November 2020.
-
Deep learning-based transformation of the H&E stain into special stains
Authors:
Kevin de Haan,
Yijie Zhang,
Jonathan E. Zuckerman,
Tairan Liu,
Anthony E. Sisk,
Miguel F. P. Diaz,
Kuang-Yu Jen,
Alexander Nobori,
Sofia Liou,
Sarah Zhang,
Rana Riahi,
Yair Rivenson,
W. Dean Wallace,
Aydogan Ozcan
Abstract:
Pathology is practiced by visual inspection of histochemically stained slides. Most commonly, the hematoxylin and eosin (H&E) stain is used in the diagnostic workflow and it is the gold standard for cancer diagnosis. However, in many cases, especially for non-neoplastic diseases, additional "special stains" are used to provide different levels of contrast and color to tissue components and allow p…
▽ More
Pathology is practiced by visual inspection of histochemically stained slides. Most commonly, the hematoxylin and eosin (H&E) stain is used in the diagnostic workflow and it is the gold standard for cancer diagnosis. However, in many cases, especially for non-neoplastic diseases, additional "special stains" are used to provide different levels of contrast and color to tissue components and allow pathologists to get a clearer diagnostic picture. In this study, we demonstrate the utility of supervised learning-based computational stain transformation from H&E to different special stains (Masson's Trichrome, periodic acid-Schiff and Jones silver stain) using tissue sections from kidney needle core biopsies. Based on evaluation by three renal pathologists, followed by adjudication by a fourth renal pathologist, we show that the generation of virtual special stains from existing H&E images improves the diagnosis in several non-neoplastic kidney diseases sampled from 58 unique subjects. A second study performed by three pathologists found that the quality of the special stains generated by the stain transformation network was statistically equivalent to those generated through standard histochemical staining. As the transformation of H&E images into special stains can be achieved within 1 min or less per patient core specimen slide, this stain-to-stain transformation framework can improve the quality of the preliminary diagnosis when additional special stains are needed, along with significant savings in time and cost, reducing the burden on healthcare system and patients.
△ Less
Submitted 12 August, 2021; v1 submitted 20 August, 2020;
originally announced August 2020.
-
A Hierarchical Framework for Long-term and Robust Deployment of Field Ground Robots in Large-Scale Farming
Authors:
Stuart Eiffert,
Nathan D. Wallace,
He Kong,
Navid Pirmarzdashti,
Salah Sukkarieh
Abstract:
Achieving long term autonomy of robots operating in dynamic environments such as farms remains a significant challenge. Arguably, the most demanding factors to achieve this are the on-board resource constraints such as energy, planning in the presence of moving individuals such as livestock and people, and handling unknown and undulating terrain. These considerations require a robot to be adaptive…
▽ More
Achieving long term autonomy of robots operating in dynamic environments such as farms remains a significant challenge. Arguably, the most demanding factors to achieve this are the on-board resource constraints such as energy, planning in the presence of moving individuals such as livestock and people, and handling unknown and undulating terrain. These considerations require a robot to be adaptive in its immediate actions in order to successfully achieve long-term, resource-efficient and robust autonomy. To achieve this, we propose a hierarchical framework that integrates a local dynamic path planner with a longer term objective based planner and advanced motion control methods, whilst taking into consideration the dynamic responses of moving individuals within the environment. The framework is motivated by and synthesizes our recent work on energy aware mission planning, path planning in dynamic environments, and receding horizon motion control. In this paper we detail the proposed framework and outline its integration on a robotic platform. We evaluate the strategy in extensive simulated trials, traversing between objective waypoints to complete tasks such as soil sampling, weeding and recharging across a dynamic environment, demonstrating its capability to robustly adapt long term mission plans in the presence of moving individuals and obstacles for real world applications such as large scale farming.
△ Less
Submitted 23 June, 2020;
originally announced June 2020.
