-
Leveraging Large Language Models to Extract Information on Substance Use Disorder Severity from Clinical Notes: A Zero-shot Learning Approach
Authors:
Maria Mahbub,
Gregory M. Dams,
Sudarshan Srinivasan,
Caitlin Rizy,
Ioana Danciu,
Jodie Trafton,
Kathryn Knight
Abstract:
Substance use disorder (SUD) poses a major concern due to its detrimental effects on health and society. SUD identification and treatment depend on a variety of factors such as severity, co-determinants (e.g., withdrawal symptoms), and social determinants of health. Existing diagnostic coding systems used by American insurance providers, like the International Classification of Diseases (ICD-10),…
▽ More
Substance use disorder (SUD) poses a major concern due to its detrimental effects on health and society. SUD identification and treatment depend on a variety of factors such as severity, co-determinants (e.g., withdrawal symptoms), and social determinants of health. Existing diagnostic coding systems used by American insurance providers, like the International Classification of Diseases (ICD-10), lack granularity for certain diagnoses, but clinicians will add this granularity (as that found within the Diagnostic and Statistical Manual of Mental Disorders classification or DSM-5) as supplemental unstructured text in clinical notes. Traditional natural language processing (NLP) methods face limitations in accurately parsing such diverse clinical language. Large Language Models (LLMs) offer promise in overcoming these challenges by adapting to diverse language patterns. This study investigates the application of LLMs for extracting severity-related information for various SUD diagnoses from clinical notes. We propose a workflow employing zero-shot learning of LLMs with carefully crafted prompts and post-processing techniques. Through experimentation with Flan-T5, an open-source LLM, we demonstrate its superior recall compared to the rule-based approach. Focusing on 11 categories of SUD diagnoses, we show the effectiveness of LLMs in extracting severity information, contributing to improved risk assessment and treatment planning for SUD patients.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
Question-Answering System Extracts Information on Injection Drug Use from Clinical Notes
Authors:
Maria Mahbub,
Ian Goethert,
Ioana Danciu,
Kathryn Knight,
Sudarshan Srinivasan,
Suzanne Tamang,
Karine Rozenberg-Ben-Dror,
Hugo Solares,
Susana Martins,
Jodie Trafton,
Edmon Begoli,
Gregory Peterson
Abstract:
Background: Injection drug use (IDU) is a dangerous health behavior that increases mortality and morbidity. Identifying IDU early and initiating harm reduction interventions can benefit individuals at risk. However, extracting IDU behaviors from patients' electronic health records (EHR) is difficult because there is no International Classification of Disease (ICD) code and the only place IDU infor…
▽ More
Background: Injection drug use (IDU) is a dangerous health behavior that increases mortality and morbidity. Identifying IDU early and initiating harm reduction interventions can benefit individuals at risk. However, extracting IDU behaviors from patients' electronic health records (EHR) is difficult because there is no International Classification of Disease (ICD) code and the only place IDU information can be indicated is unstructured free-text clinical notes. Although natural language processing can efficiently extract this information from unstructured data, there are no validated tools. Methods: To address this gap in clinical information, we design and demonstrate a question-answering (QA) framework to extract information on IDU from clinical notes. Our framework involves two main steps: (1) generating a gold-standard QA dataset and (2) develo** and testing the QA model. We utilize 2323 clinical notes of 1145 patients sourced from the VA Corporate Data Warehouse to construct the gold-standard dataset for develo** and evaluating the QA model. We also demonstrate the QA model's ability to extract IDU-related information on temporally out-of-distribution data. Results: Here we show that for a strict match between gold-standard and predicted answers, the QA model achieves 51.65% F1 score. For a relaxed match between the gold-standard and predicted answers, the QA model obtains 78.03% F1 score, along with 85.38% Precision and 79.02% Recall scores. Moreover, the QA model demonstrates consistent performance when subjected to temporally out-of-distribution data. Conclusions: Our study introduces a QA framework designed to extract IDU information from clinical notes, aiming to enhance the accurate and efficient detection of people who inject drugs, extract relevant information, and ultimately facilitate informed patient care.
△ Less
Submitted 28 December, 2023; v1 submitted 15 May, 2023;
originally announced May 2023.
-
2nd Workshop on Cognitive Architectures for Social Human-Robot Interaction 2016 (CogArch4sHRI 2016)
Authors:
Paul Baxter,
J. Gregory Trafton,
Severin Lemaignan
Abstract:
This volume is the proceedings of the 2nd workshop on Cognitive Architectures for Social Human-Robot Interaction, held at the ACM/IEEE HRI 2016 conference, which took place on Monday 7th March 2016, in Christchurch, New Zealand.
Organised by Paul Baxter (Plymouth University, U.K.), J. Gregory Trafton (Naval Research Laboratory, USA), and Severin Lemaignan (Plymouth University, U.K.).
This volume is the proceedings of the 2nd workshop on Cognitive Architectures for Social Human-Robot Interaction, held at the ACM/IEEE HRI 2016 conference, which took place on Monday 7th March 2016, in Christchurch, New Zealand.
Organised by Paul Baxter (Plymouth University, U.K.), J. Gregory Trafton (Naval Research Laboratory, USA), and Severin Lemaignan (Plymouth University, U.K.).
△ Less
Submitted 4 February, 2016;
originally announced February 2016.
