-
Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization
Authors:
Dave Van Veen,
Cara Van Uden,
Louis Blankemeier,
Jean-Benoit Delbrouck,
Asad Aali,
Christian Bluethgen,
Anuj Pareek,
Malgorzata Polacin,
Eduardo Pontes Reis,
Anna Seehofnerova,
Nidhi Rohatgi,
Poonam Hosamani,
William Collins,
Neera Ahuja,
Curtis P. Langlotz,
Jason Hom,
Sergios Gatidis,
John Pauly,
Akshay S. Chaudhari
Abstract:
Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP), their effectiveness on a diverse range of clinical summarization tasks remains unproven. In this study, we apply adaptation methods to eight LLMs,…
▽ More
Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP), their effectiveness on a diverse range of clinical summarization tasks remains unproven. In this study, we apply adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. Quantitative assessments with syntactic, semantic, and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with ten physicians evaluates summary completeness, correctness, and conciseness; in a majority of cases, summaries from our best adapted LLMs are either equivalent (45%) or superior (36%) compared to summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.
△ Less
Submitted 11 April, 2024; v1 submitted 14 September, 2023;
originally announced September 2023.
-
Exploring the Versatility of Zero-Shot CLIP for Interstitial Lung Disease Classification
Authors:
Cara Van Uden,
Christian Bluethgen,
Maayane Attias,
Malgorzata Polacin,
Haiwei Henry Guo,
Neha Simha,
Rishi Raj,
Curtis Langlotz
Abstract:
Interstitial lung diseases (ILD) present diagnostic challenges due to their varied manifestations and overlap** imaging features. To address this, we propose a machine learning approach that utilizes CLIP, a multimodal (image and text) self-supervised model, for ILD classification. We extensively integrate zero-shot CLIP throughout our workflow, starting from the initial extraction of image patc…
▽ More
Interstitial lung diseases (ILD) present diagnostic challenges due to their varied manifestations and overlap** imaging features. To address this, we propose a machine learning approach that utilizes CLIP, a multimodal (image and text) self-supervised model, for ILD classification. We extensively integrate zero-shot CLIP throughout our workflow, starting from the initial extraction of image patches from volumetric CT scans and proceeding to ILD classification using "patch montages". Furthermore, we investigate how domain adaptive pretraining (DAPT) CLIP with task-specific images (CT "patch montages" extracted with ILD-specific prompts for CLIP) and/or text (lung-specific sections of radiology reports) affects downstream ILD classification performance. By leveraging CLIP-extracted "patch montages" and DAPT, we achieve strong zero-shot ILD classification results, including an AUROC of 0.893, without the need for any labeled training data. This work highlights the versatility and potential of multimodal models like CLIP for medical image classification tasks where labeled data is scarce.
△ Less
Submitted 12 September, 2023; v1 submitted 1 June, 2023;
originally announced June 2023.
-
RadAdapt: Radiology Report Summarization via Lightweight Domain Adaptation of Large Language Models
Authors:
Dave Van Veen,
Cara Van Uden,
Maayane Attias,
Anuj Pareek,
Christian Bluethgen,
Malgorzata Polacin,
Wah Chiu,
Jean-Benoit Delbrouck,
Juan Manuel Zambrano Chaves,
Curtis P. Langlotz,
Akshay S. Chaudhari,
John Pauly
Abstract:
We systematically investigate lightweight strategies to adapt large language models (LLMs) for the task of radiology report summarization (RRS). Specifically, we focus on domain adaptation via pretraining (on natural language, biomedical text, or clinical text) and via discrete prompting or parameter-efficient fine-tuning. Our results consistently achieve best performance by maximally adapting to…
▽ More
We systematically investigate lightweight strategies to adapt large language models (LLMs) for the task of radiology report summarization (RRS). Specifically, we focus on domain adaptation via pretraining (on natural language, biomedical text, or clinical text) and via discrete prompting or parameter-efficient fine-tuning. Our results consistently achieve best performance by maximally adapting to the task via pretraining on clinical text and fine-tuning on RRS examples. Importantly, this method fine-tunes a mere 0.32% of parameters throughout the model, in contrast to end-to-end fine-tuning (100% of parameters). Additionally, we study the effect of in-context examples and out-of-distribution (OOD) training before concluding with a radiologist reader study and qualitative analysis. Our findings highlight the importance of domain adaptation in RRS and provide valuable insights toward develo** effective natural language processing solutions for clinical tasks.
△ Less
Submitted 20 July, 2023; v1 submitted 1 May, 2023;
originally announced May 2023.
-
RoentGen: Vision-Language Foundation Model for Chest X-ray Generation
Authors:
Pierre Chambon,
Christian Bluethgen,
Jean-Benoit Delbrouck,
Rogier Van der Sluijs,
Małgorzata Połacin,
Juan Manuel Zambrano Chaves,
Tanishq Mathew Abraham,
Shivanshu Purohit,
Curtis P. Langlotz,
Akshay Chaudhari
Abstract:
Multimodal models trained on large natural image-text pair datasets have exhibited astounding abilities in generating high-quality images. Medical imaging data is fundamentally different to natural images, and the language used to succinctly capture relevant details in medical data uses a different, narrow but semantically rich, domain-specific vocabulary. Not surprisingly, multi-modal models trai…
▽ More
Multimodal models trained on large natural image-text pair datasets have exhibited astounding abilities in generating high-quality images. Medical imaging data is fundamentally different to natural images, and the language used to succinctly capture relevant details in medical data uses a different, narrow but semantically rich, domain-specific vocabulary. Not surprisingly, multi-modal models trained on natural image-text pairs do not tend to generalize well to the medical domain. Develo** generative imaging models faithfully representing medical concepts while providing compositional diversity could mitigate the existing paucity of high-quality, annotated medical imaging datasets. In this work, we develop a strategy to overcome the large natural-medical distributional shift by adapting a pre-trained latent diffusion model on a corpus of publicly available chest x-rays (CXR) and their corresponding radiology (text) reports. We investigate the model's ability to generate high-fidelity, diverse synthetic CXR conditioned on text prompts. We assess the model outputs quantitatively using image quality metrics, and evaluate image quality and text-image alignment by human domain experts. We present evidence that the resulting model (RoentGen) is able to create visually convincing, diverse synthetic CXR images, and that the output can be controlled to a new extent by using free-form text prompts including radiology-specific language. Fine-tuning this model on a fixed training set and using it as a data augmentation method, we measure a 5% improvement of a classifier trained jointly on synthetic and real images, and a 3% improvement when trained on a larger but purely synthetic training set. Finally, we observe that this fine-tuning distills in-domain knowledge in the text-encoder and can improve its representation capabilities of certain diseases like pneumothorax by 25%.
△ Less
Submitted 23 November, 2022;
originally announced November 2022.
