-
Towards Reducing Diagnostic Errors with Interpretable Risk Prediction
Authors:
Denis Jered McInerney,
William Dickinson,
Lucy C. Flynn,
Andrea C. Young,
Geoffrey S. Young,
Jan-Willem van de Meent,
Byron C. Wallace
Abstract:
Many diagnostic errors occur because clinicians cannot easily access relevant information in patient Electronic Health Records (EHRs). In this work we propose a method to use LLMs to identify pieces of evidence in patient EHR data that indicate increased or decreased risk of specific diagnoses; our ultimate aim is to increase access to evidence and reduce diagnostic errors. In particular, we propo…
▽ More
Many diagnostic errors occur because clinicians cannot easily access relevant information in patient Electronic Health Records (EHRs). In this work we propose a method to use LLMs to identify pieces of evidence in patient EHR data that indicate increased or decreased risk of specific diagnoses; our ultimate aim is to increase access to evidence and reduce diagnostic errors. In particular, we propose a Neural Additive Model to make predictions backed by evidence with individualized risk estimates at time-points where clinicians are still uncertain, aiming to specifically mitigate delays in diagnosis and errors stemming from an incomplete differential. To train such a model, it is necessary to infer temporally fine-grained retrospective labels of eventual "true" diagnoses. We do so with LLMs, to ensure that the input text is from before a confident diagnosis can be made. We use an LLM to retrieve an initial pool of evidence, but then refine this set of evidence according to correlations learned by the model. We conduct an in-depth evaluation of the usefulness of our approach by simulating how it might be used by a clinician to decide between a pre-defined list of differential diagnoses.
△ Less
Submitted 19 March, 2024; v1 submitted 15 February, 2024;
originally announced February 2024.
-
Everyday Uses of Music Listening and Music Technologies by Caregivers and People with Dementia: Survey and Focus Group Study
Authors:
Dianna Vidas,
Romina Carrasco,
Ryan M. Kelly,
Jenny Waycott,
Jeanette Tamplin,
Kate McMahon,
Libby M. Flynn,
Phoebe A. Stretton-Smith,
Tanara Vieira Sousa,
Felicity A. Baker
Abstract:
Music is a valuable non-pharmacological tool that provides benefits for people with dementia, and there is interest in designing technologies to support music use in dementia care. To ensure music technologies are appropriately designed for supporting caregivers and people living with dementia, there remains a need to better understand how music is currently used in everyday care at home. We aimed…
▽ More
Music is a valuable non-pharmacological tool that provides benefits for people with dementia, and there is interest in designing technologies to support music use in dementia care. To ensure music technologies are appropriately designed for supporting caregivers and people living with dementia, there remains a need to better understand how music is currently used in everyday care at home. We aimed to understand how people with dementia and their caregivers use music technologies in everyday caring, as well as challenges they experience using music and technology. This study used a mixed methods design. A survey was completed by 77 caregivers and people with dementia to understand their use of music and technology. Of these, 18 survey respondents (12 family caregivers, 6 people living with dementia) participated in focus groups about their experiences of using music and technology in care. Transcripts were analysed with reflexive thematic analysis. Most survey respondents used music often in their daily lives, reporting a range of music technologies such as CDs, radio, and streaming. Focus groups highlighted benefits and challenges of music technologies in everyday care. Participants used music and music technologies to regulate mood, provide joy, facilitate social connection, encourage reminiscence, provide continuity before and after diagnosis, and to make caregiving easier. Challenges of using music technology in care included difficulties staying up to date with evolving technology, and low self-efficacy for technology use expressed by people living with dementia. Evidently, people living with dementia and their caregivers use music technologies to support their everyday care needs. Results suggest opportunities to design technologies enabling easier access to music and supporting people living with dementia with recreational and therapeutic music listening and music-based activities.
△ Less
Submitted 1 February, 2024;
originally announced February 2024.
-
A Study of Static Warning Cascading Tools (Experience Paper)
Authors:
Xiuyuan Guo,
Ashwin Kallingal Joshy,
Benjamin Steenhoek,
Wei Le,
Lori Flynn
Abstract:
Static analysis is widely used for software assurance. However, static analysis tools can report an overwhelming number of warnings, many of which are false positives. Applying static analysis to a new version, a large number of warnings can be only relevant to the old version. Inspecting these warnings is a waste of time and can prevent developers from finding the new bugs in the new version. In…
▽ More
Static analysis is widely used for software assurance. However, static analysis tools can report an overwhelming number of warnings, many of which are false positives. Applying static analysis to a new version, a large number of warnings can be only relevant to the old version. Inspecting these warnings is a waste of time and can prevent developers from finding the new bugs in the new version. In this paper, we report the challenges of cascading warnings generated from two versions of programs. We investigated program differencing tools and extend them to perform warning cascading automatically. Specifically, we used textual based diff tool, namely SCALe, abstract syntax tree (AST) based diff tool, namely GumTree, and control flow graph (CFG) based diff tool, namely Hydrogen. We reported our experience of applying these tools and hopefully our findings can provide developers understandings of pros and cons of each approach. In our evaluation, we used 96 pairs of benchmark programs for which we know ground-truth bugs and fixes as well as 12 pairs of real-world open-source projects. Our tools and data are available at https: //github.com/WarningCas/WarningCascading_Data.
△ Less
Submitted 3 May, 2023;
originally announced May 2023.
-
Test Suites as a Source of Training Data for Static Analysis Alert Classifiers
Authors:
Lori Flynn,
William Snavely,
Zachary Kurtz
Abstract:
Flaw-finding static analysis tools typically generate large volumes of code flaw alerts including many false positives. To save on human effort to triage these alerts, a significant body of work attempts to use machine learning to classify and prioritize alerts. Identifying a useful set of training data, however, remains a fundamental challenge in develo** such classifiers in many contexts. We p…
▽ More
Flaw-finding static analysis tools typically generate large volumes of code flaw alerts including many false positives. To save on human effort to triage these alerts, a significant body of work attempts to use machine learning to classify and prioritize alerts. Identifying a useful set of training data, however, remains a fundamental challenge in develo** such classifiers in many contexts. We propose using static analysis test suites (i.e., repositories of "benchmark" programs that are purpose-built to test coverage and precision of static analysis tools) as a novel source of training data. In a case study, we generated a large quantity of alerts by executing various static analyzers on the Juliet C/C++ test suite, and we automatically derived ground truth labels for these alerts by referencing the Juliet test suite metadata. Finally, we used this data to train classifiers to predict whether an alert is a false positive. Our classifiers obtained high precision (90.2%) and recall (88.2%) for a large number of code flaw types on a hold-out test set. This preliminary result suggests that pre-training classifiers on test suite data could help to jumpstart static analysis alert classification in data-limited contexts.
△ Less
Submitted 7 May, 2021;
originally announced May 2021.
