-
Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging
Authors:
Poojitha Thota,
Jai Prakash Veerla,
Partha Sai Guttikonda,
Mohammad S. Nasr,
Shirin Nilizadeh,
Jacob M. Luber
Abstract:
In the context of medical artificial intelligence, this study explores the vulnerabilities of the Pathology Language-Image Pretraining (PLIP) model, a Vision Language Foundation model, under targeted attacks. Leveraging the Kather Colon dataset with 7,180 H&E images across nine tissue types, our investigation employs Projected Gradient Descent (PGD) adversarial perturbation attacks to induce miscl…
▽ More
In the context of medical artificial intelligence, this study explores the vulnerabilities of the Pathology Language-Image Pretraining (PLIP) model, a Vision Language Foundation model, under targeted attacks. Leveraging the Kather Colon dataset with 7,180 H&E images across nine tissue types, our investigation employs Projected Gradient Descent (PGD) adversarial perturbation attacks to induce misclassifications intentionally. The outcomes reveal a 100% success rate in manipulating PLIP's predictions, underscoring its susceptibility to adversarial perturbations. The qualitative analysis of adversarial examples delves into the interpretability challenges, shedding light on nuanced changes in predictions induced by adversarial manipulations. These findings contribute crucial insights into the interpretability, domain adaptation, and trustworthiness of Vision Language Models in medical imaging. The study emphasizes the pressing need for robust defenses to ensure the reliability of AI models. The source codes for this experiment can be found at https://github.com/jaiprakash1824/VLM_Adv_Attack.
△ Less
Submitted 7 May, 2024; v1 submitted 4 January, 2024;
originally announced January 2024.
-
Real-Time Diagnostic Integrity Meets Efficiency: A Novel Platform-Agnostic Architecture for Physiological Signal Compression
Authors:
Neel R Vora,
Amir Hajighasemi,
Cody T. Reynolds,
Amirmohammad Radmehr,
Mohamed Mohamed,
Jillur Rahman Saurav,
Abdul Aziz,
Jai Prakash Veerla,
Mohammad S Nasr,
Hayden Lotspeich,
Partha Sai Guttikonda,
Thuong Pham,
Aarti Darji,
Parisa Boodaghi Malidarreh,
Helen H Shang,
Jay Harvey,
Kan Ding,
Phuc Nguyen,
Jacob M Luber
Abstract:
Head-based signals such as EEG, EMG, EOG, and ECG collected by wearable systems will play a pivotal role in clinical diagnosis, monitoring, and treatment of important brain disorder diseases.
However, the real-time transmission of the significant corpus physiological signals over extended periods consumes substantial power and time, limiting the viability of battery-dependent physiological monit…
▽ More
Head-based signals such as EEG, EMG, EOG, and ECG collected by wearable systems will play a pivotal role in clinical diagnosis, monitoring, and treatment of important brain disorder diseases.
However, the real-time transmission of the significant corpus physiological signals over extended periods consumes substantial power and time, limiting the viability of battery-dependent physiological monitoring wearables.
This paper presents a novel deep-learning framework employing a variational autoencoder (VAE) for physiological signal compression to reduce wearables' computational complexity and energy consumption.
Our approach achieves an impressive compression ratio of 1:293 specifically for spectrogram data, surpassing state-of-the-art compression techniques such as JPEG2000, H.264, Direct Cosine Transform (DCT), and Huffman Encoding, which do not excel in handling physiological signals.
We validate the efficacy of the compressed algorithms using collected physiological signals from real patients in the Hospital and deploy the solution on commonly used embedded AI chips (i.e., ARM Cortex V8 and Jetson Nano). The proposed framework achieves a 91% seizure detection accuracy using XGBoost, confirming the approach's reliability, practicality, and scalability.
△ Less
Submitted 4 January, 2024; v1 submitted 19 December, 2023;
originally announced December 2023.
-
Histopathology Slide Indexing and Search: Are We There Yet?
Authors:
Helen H. Shang,
Mohammad Sadegh Nasr,
Jai Prakash Veerla,
Parisa Boodaghi Malidarreh,
MD Jillur Rahman Saurav,
Amir Hajighasemi,
Manfred Huber,
Chace Moleta,
Jitin Makker,
Jacob M. Luber
Abstract:
The search and retrieval of digital histopathology slides is an important task that has yet to be solved. In this case study, we investigate the clinical readiness of three state-of-the-art histopathology slide search engines, Yottixel, SISH, and RetCCL, on three patients with solid tumors. We provide a qualitative assessment of each model's performance in providing retrieval results that are reli…
▽ More
The search and retrieval of digital histopathology slides is an important task that has yet to be solved. In this case study, we investigate the clinical readiness of three state-of-the-art histopathology slide search engines, Yottixel, SISH, and RetCCL, on three patients with solid tumors. We provide a qualitative assessment of each model's performance in providing retrieval results that are reliable and useful to pathologists. We found that all three image search engines fail to produce consistently reliable results and have difficulties in capturing granular and subtle features of malignancy, limiting their diagnostic accuracy. Based on our findings, we also propose a minimal set of requirements to further advance the development of accurate and reliable histopathology image search engines for successful clinical adoption.
△ Less
Submitted 4 January, 2024; v1 submitted 29 June, 2023;
originally announced June 2023.
-
The State of Applying Artificial Intelligence to Tissue Imaging for Cancer Research and Early Detection
Authors:
Michael Robben,
Amir Hajighasemi,
Mohammad Sadegh Nasr,
Jai Prakesh Veerla,
Anne M. Alsup,
Biraaj Rout,
Helen H. Shang,
Kelli Fowlds,
Parisa Boodaghi Malidarreh,
Paul Koomey,
MD Jillur Rahman Saurav,
Jacob M. Luber
Abstract:
Artificial intelligence represents a new frontier in human medicine that could save more lives and reduce the costs, thereby increasing accessibility. As a consequence, the rate of advancement of AI in cancer medical imaging and more particularly tissue pathology has exploded, opening it to ethical and technical questions that could impede its adoption into existing systems. In order to chart the…
▽ More
Artificial intelligence represents a new frontier in human medicine that could save more lives and reduce the costs, thereby increasing accessibility. As a consequence, the rate of advancement of AI in cancer medical imaging and more particularly tissue pathology has exploded, opening it to ethical and technical questions that could impede its adoption into existing systems. In order to chart the path of AI in its application to cancer tissue imaging, we review current work and identify how it can improve cancer pathology diagnostics and research. In this review, we identify 5 core tasks that models are developed for, including regression, classification, segmentation, generation, and compression tasks. We address the benefits and challenges that such methods face, and how they can be adapted for use in cancer prevention and treatment. The studies looked at in this paper represent the beginning of this field and future experiments will build on the foundations that we highlight.
△ Less
Submitted 29 June, 2023;
originally announced June 2023.
-
Multimodal Pathology Image Search Between H&E Slides and Multiplexed Immunofluorescent Images
Authors:
Amir Hajighasemi,
MD Jillur Rahman Saurav,
Mohammad S Nasr,
Jai Prakash Veerla,
Aarti Darji,
Parisa Boodaghi Malidarreh,
Michael Robben,
Helen H Shang,
Jacob M Luber
Abstract:
We present an approach for multimodal pathology image search, using dynamic time war** (DTW) on Variational Autoencoder (VAE) latent space that is fed into a ranked choice voting scheme to retrieve multiplexed immunofluorescent imaging (mIF) that is most similar to a query H&E slide. Through training the VAE and applying DTW, we align and compare mIF and H&E slides. Our method improves different…
▽ More
We present an approach for multimodal pathology image search, using dynamic time war** (DTW) on Variational Autoencoder (VAE) latent space that is fed into a ranked choice voting scheme to retrieve multiplexed immunofluorescent imaging (mIF) that is most similar to a query H&E slide. Through training the VAE and applying DTW, we align and compare mIF and H&E slides. Our method improves differential diagnosis and therapeutic decisions by integrating morphological H&E data with immunophenoty** from mIF, providing clinicians a rich perspective of disease states. This facilitates an understanding of the spatial relationships in tissue samples and could revolutionize the diagnostic process, enhancing precision and enabling personalized therapy selection. Our technique demonstrates feasibility using colorectal cancer and healthy tonsil samples. An exhaustive ablation study was conducted on a search engine designed to explore the correlation between multiplexed Immunofluorescence (mIF) and Hematoxylin and Eosin (H&E) staining, in order to validate its ability to map these distinct modalities into a unified vector space. Despite extreme class imbalance, the system demonstrated robustness and utility by returning similar results across various data features, which suggests potential for future use in multimodal histopathology data analysis.
△ Less
Submitted 11 June, 2023;
originally announced June 2023.