-
Knowledge-grounded Adaptation Strategy for Vision-language Models: Building Unique Case-set for Screening Mammograms for Residents Training
Authors:
Aisha Urooj Khan,
John Garrett,
Tyler Bradshaw,
Lonie Salkowski,
Jiwoong Jason Jeong,
Amara Tariq,
Imon Banerjee
Abstract:
A visual-language model (VLM) pre-trained on natural images and text pairs poses a significant barrier when applied to medical contexts due to domain shift. Yet, adapting or fine-tuning these VLMs for medical use presents considerable hurdles, including domain misalignment, limited access to extensive datasets, and high-class imbalances. Hence, there is a pressing need for strategies to effectivel…
▽ More
A visual-language model (VLM) pre-trained on natural images and text pairs poses a significant barrier when applied to medical contexts due to domain shift. Yet, adapting or fine-tuning these VLMs for medical use presents considerable hurdles, including domain misalignment, limited access to extensive datasets, and high-class imbalances. Hence, there is a pressing need for strategies to effectively adapt these VLMs to the medical domain, as such adaptations would prove immensely valuable in healthcare applications. In this study, we propose a framework designed to adeptly tailor VLMs to the medical domain, employing selective sampling and hard-negative mining techniques for enhanced performance in retrieval tasks. We validate the efficacy of our proposed approach by implementing it across two distinct VLMs: the in-domain VLM (MedCLIP) and out-of-domain VLMs (ALBEF). We assess the performance of these models both in their original off-the-shelf state and after undergoing our proposed training strategies, using two extensive datasets containing mammograms and their corresponding reports. Our evaluation spans zero-shot, few-shot, and supervised scenarios. Through our approach, we observe a notable enhancement in Recall@K performance for the image-text retrieval task.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
The EMory BrEast imaging Dataset (EMBED): A Racially Diverse, Granular Dataset of 3.5M Screening and Diagnostic Mammograms
Authors:
Jiwoong J. Jeong,
Brianna L. Vey,
Ananth Reddy,
Thomas Kim,
Thiago Santos,
Ramon Correa,
Raman Dutt,
Marina Mosunjac,
Gabriela Oprea-Ilies,
Geoffrey Smith,
Minjae Woo,
Christopher R. McAdams,
Mary S. Newell,
Imon Banerjee,
Judy Gichoya,
Hari Trivedi
Abstract:
Develo** and validating artificial intelligence models in medical imaging requires datasets that are large, granular, and diverse. To date, the majority of publicly available breast imaging datasets lack in one or more of these areas. Models trained on these data may therefore underperform on patient populations or pathologies that have not previously been encountered. The EMory BrEast imaging D…
▽ More
Develo** and validating artificial intelligence models in medical imaging requires datasets that are large, granular, and diverse. To date, the majority of publicly available breast imaging datasets lack in one or more of these areas. Models trained on these data may therefore underperform on patient populations or pathologies that have not previously been encountered. The EMory BrEast imaging Dataset (EMBED) addresses these gaps by providing 3650,000 2D and DBT screening and diagnostic mammograms for 116,000 women divided equally between White and African American patients. The dataset also contains 40,000 annotated lesions linked to structured imaging descriptors and 61 ground truth pathologic outcomes grouped into six severity classes. Our goal is to share this dataset with research partners to aid in development and validation of breast AI models that will serve all patients fairly and help decrease bias in medical AI.
△ Less
Submitted 8 February, 2022;
originally announced February 2022.
-
Two-step adversarial debiasing with partial learning -- medical image case-studies
Authors:
Ramon Correa,
Jiwoong Jason Jeong,
Bhavik Patel,
Hari Trivedi,
Judy W. Gichoya,
Imon Banerjee
Abstract:
The use of artificial intelligence (AI) in healthcare has become a very active research area in the last few years. While significant progress has been made in image classification tasks, only a few AI methods are actually being deployed in hospitals. A major hurdle in actively using clinical AI models currently is the trustworthiness of these models. More often than not, these complex models are…
▽ More
The use of artificial intelligence (AI) in healthcare has become a very active research area in the last few years. While significant progress has been made in image classification tasks, only a few AI methods are actually being deployed in hospitals. A major hurdle in actively using clinical AI models currently is the trustworthiness of these models. More often than not, these complex models are black boxes in which promising results are generated. However, when scrutinized, these models begin to reveal implicit biases during the decision making, such as detecting race and having bias towards ethnic groups and subpopulations. In our ongoing study, we develop a two-step adversarial debiasing approach with partial learning that can reduce the racial disparity while preserving the performance of the targeted task. The methodology has been evaluated on two independent medical image case-studies - chest X-ray and mammograms, and showed promises in bias reduction while preserving the targeted performance.
△ Less
Submitted 16 November, 2021;
originally announced November 2021.
-
Data Analysis: Communicating with Offshore Vendors using Instant Messaging Services
Authors:
Jongkil Jay Jeong
Abstract:
The purpose of this study is to find whether the choice of correct analytic process is effective to derive a meaningful and correct conclusion from the vast amount of information. For this purpose, I designed an analytic framework to investigate the importance of effective communication on the success of IT business. Through an detailed analysis of chat conversations between a outsource service pr…
▽ More
The purpose of this study is to find whether the choice of correct analytic process is effective to derive a meaningful and correct conclusion from the vast amount of information. For this purpose, I designed an analytic framework to investigate the importance of effective communication on the success of IT business. Through an detailed analysis of chat conversations between a outsource service provider and client, this study found evidence to suggest that the language used in instant messaging environments between clients & offshore providers was highly fragmented and broken, but both the client and offshore provider did not seemed to be impacted by these anomalies.
△ Less
Submitted 7 August, 2021;
originally announced August 2021.
-
Success in IT offshoring: Does it depend on the location or the company?
Authors:
Jongkil Jay Jeong
Abstract:
Many companies are now looking towards offshore vendors to fulfill their outsourcing requirements. With this growth, we have seen particular countries such as India dominate the offshoring market, and this paper will examine what type of role the reputation of a particular offshoring country has on the decision making process of firms looking to offshore.
Many companies are now looking towards offshore vendors to fulfill their outsourcing requirements. With this growth, we have seen particular countries such as India dominate the offshoring market, and this paper will examine what type of role the reputation of a particular offshoring country has on the decision making process of firms looking to offshore.
△ Less
Submitted 7 August, 2021;
originally announced August 2021.
