-
EHRmonize: A Framework for Medical Concept Abstraction from Electronic Health Records using Large Language Models
Authors:
João Matos,
Jack Gallifant,
Jian Pei,
A. Ian Wong
Abstract:
Electronic health records (EHRs) contain vast amounts of complex data, but harmonizing and processing this information remains a challenging and costly task requiring significant clinical expertise. While large language models (LLMs) have shown promise in various healthcare applications, their potential for abstracting medical concepts from EHRs remains largely unexplored. We introduce EHRmonize,…
▽ More
Electronic health records (EHRs) contain vast amounts of complex data, but harmonizing and processing this information remains a challenging and costly task requiring significant clinical expertise. While large language models (LLMs) have shown promise in various healthcare applications, their potential for abstracting medical concepts from EHRs remains largely unexplored. We introduce EHRmonize, a framework leveraging LLMs to abstract medical concepts from EHR data. Our study uses medication data from two real-world EHR databases to evaluate five LLMs on two free-text extraction and six binary classification tasks across various prompting strategies. GPT-4o's with 10-shot prompting achieved the highest performance in all tasks, accompanied by Claude-3.5-Sonnet in a subset of tasks. GPT-4o achieved an accuracy of 97% in identifying generic route names, 82% for generic drug names, and 100% in performing binary classification of antibiotics. While EHRmonize significantly enhances efficiency, reducing annotation time by an estimated 60%, we emphasize that clinician oversight remains essential. Our framework, available as a Python package, offers a promising tool to assist clinicians in EHR data abstraction, potentially accelerating healthcare research and improving data harmonization processes.
△ Less
Submitted 28 June, 2024;
originally announced July 2024.
-
Multi-Camera Visual-Inertial Simultaneous Localization and Map** for Autonomous Valet Parking
Authors:
Marcus Abate,
Ariel Schwartz,
Xue Iuan Wong,
Wangdong Luo,
Rotem Littman,
Marc Klinger,
Lars Kuhnert,
Douglas Blue,
Luca Carlone
Abstract:
Localization and map** are key capabilities for self-driving vehicles. In this paper, we build on Kimera and extend it to use multiple cameras as well as external (eg wheel) odometry sensors, to obtain accurate and robust odometry estimates in real-world problems. Additionally, we propose an effective scheme for closing loops that circumvents the drawbacks of common alternatives based on the Per…
▽ More
Localization and map** are key capabilities for self-driving vehicles. In this paper, we build on Kimera and extend it to use multiple cameras as well as external (eg wheel) odometry sensors, to obtain accurate and robust odometry estimates in real-world problems. Additionally, we propose an effective scheme for closing loops that circumvents the drawbacks of common alternatives based on the Perspective-n-Point method and also works with a single monocular camera. Finally, we develop a method for dense 3D map** of the free space that combines a segmentation network for free-space detection with a homography-based dense map** technique. We test our system on photo-realistic simulations and on several real datasets collected on a car prototype developed by the Ford Motor Company, spanning both indoor and outdoor parking scenarios. Our multi-camera system is shown to outperform state-of-the art open-source visual-inertial-SLAM pipelines (Vins-Fusion, ORB-SLAM3), and exhibits an average trajectory error under 1% of the trajectory length across more than 8km of distance traveled (combined across all datasets). A video showcasing the system is available at: youtu.be/H8CpzDpXOI8.
△ Less
Submitted 11 January, 2024; v1 submitted 25 April, 2023;
originally announced April 2023.
-
Reanalyzing L2 Preposition Learning with Bayesian Mixed Effects and a Pretrained Language Model
Authors:
Jakob Prange,
Man Ho Ivy Wong
Abstract:
We use both Bayesian and neural models to dissect a data set of Chinese learners' pre- and post-interventional responses to two tests measuring their understanding of English prepositions. The results mostly replicate previous findings from frequentist analyses and newly reveal crucial interactions between student ability, task type, and stimulus sentence. Given the sparsity of the data as well as…
▽ More
We use both Bayesian and neural models to dissect a data set of Chinese learners' pre- and post-interventional responses to two tests measuring their understanding of English prepositions. The results mostly replicate previous findings from frequentist analyses and newly reveal crucial interactions between student ability, task type, and stimulus sentence. Given the sparsity of the data as well as high diversity among learners, the Bayesian method proves most useful; but we also see potential in using language model probabilities as predictors of grammaticality and learnability.
△ Less
Submitted 23 May, 2023; v1 submitted 16 February, 2023;
originally announced February 2023.
-
Radio Galaxy Zoo: Using semi-supervised learning to leverage large unlabelled data-sets for radio galaxy classification under data-set shift
Authors:
Inigo V. Slijepcevic,
Anna M. M. Scaife,
Mike Walmsley,
Micah Bowles,
Ivy Wong,
Stanislav S. Shabala,
Hongming Tang
Abstract:
In this work we examine the classification accuracy and robustness of a state-of-the-art semi-supervised learning (SSL) algorithm applied to the morphological classification of radio galaxies. We test if SSL with fewer labels can achieve test accuracies comparable to the supervised state-of-the-art and whether this holds when incorporating previously unseen data. We find that for the radio galaxy…
▽ More
In this work we examine the classification accuracy and robustness of a state-of-the-art semi-supervised learning (SSL) algorithm applied to the morphological classification of radio galaxies. We test if SSL with fewer labels can achieve test accuracies comparable to the supervised state-of-the-art and whether this holds when incorporating previously unseen data. We find that for the radio galaxy classification problem considered, SSL provides additional regularisation and outperforms the baseline test accuracy. However, in contrast to model performance metrics reported on computer science benchmarking data-sets, we find that improvement is limited to a narrow range of label volumes, with performance falling off rapidly at low label volumes. Additionally, we show that SSL does not improve model calibration, regardless of whether classification is improved. Moreover, we find that when different underlying catalogues drawn from the same radio survey are used to provide the labelled and unlabelled data-sets required for SSL, a significant drop in classification performance is observered, highlighting the difficulty of applying SSL techniques under dataset shift. We show that a class-imbalanced unlabelled data pool negatively affects performance through prior probability shift, which we suggest may explain this performance drop, and that using the Frechet Distance between labelled and unlabelled data-sets as a measure of data-set shift can provide a prediction of model performance, but that for typical radio galaxy data-sets with labelled sample volumes of O(1000), the sample variance associated with this technique is high and the technique is in general not sufficiently robust to replace a train-test cycle.
△ Less
Submitted 4 May, 2022; v1 submitted 19 April, 2022;
originally announced April 2022.
-
Benchmarking emergency department triage prediction models with machine learning and large public electronic health records
Authors:
Feng Xie,
Jun Zhou,
** Wee Lee,
Mingrui Tan,
Siqi Li,
Logasan S/O Rajnthern,
Marcel Lucas Chee,
Bibhas Chakraborty,
An-Kwok Ian Wong,
Alon Dagan,
Marcus Eng Hock Ong,
Fei Gao,
Nan Liu
Abstract:
The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have acce…
▽ More
The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop predictive models and decision support systems to address these challenges. To date, however, there are no widely accepted benchmark ED triage prediction models based on large-scale public EHR data. An open-source benchmarking platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. In this paper, based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we developed a publicly available benchmark suite for ED triage predictive models and created a benchmark dataset that contains over 400,000 ED visits from 2011 to 2019. We introduced three ED-based outcomes (hospitalization, critical outcomes, and 72-hour ED reattendance) and implemented a variety of popular methodologies, ranging from machine learning methods to clinical scoring systems. We evaluated and compared the performance of these methods against benchmark tasks. Our codes are open-source, allowing anyone with MIMIC-IV-ED data access to perform the same steps in data processing, benchmark model building, and experiments. This study provides future researchers with insights, suggestions, and protocols for managing raw data and develo** risk triaging tools for emergency care.
△ Less
Submitted 20 March, 2022; v1 submitted 22 November, 2021;
originally announced November 2021.
-
Agile Information System Development Organizations Transforming to Large-Scale Collaboration
Authors:
Marius Mikalsen,
Nils Brede Moe,
Sut I Wong,
Viktoria Stray
Abstract:
We report findings from a case study of a large agile information systems development (ISD) organization`s sudden transformation to distributed, digital work in the context of the Covid-19 pandemic. It seeks to understand how knowledge creation and sharing changes. The findings show various forms of distance being introduced, digital tool usage, increased task orientation, and variations across te…
▽ More
We report findings from a case study of a large agile information systems development (ISD) organization`s sudden transformation to distributed, digital work in the context of the Covid-19 pandemic. It seeks to understand how knowledge creation and sharing changes. The findings show various forms of distance being introduced, digital tool usage, increased task orientation, and variations across teams. To analyze the findings, we use the concepts of large-scale collaborations and sociability. Large-scale collaboration offers a socio-technical perspective on tackling distributed knowledge sharing and creation in the presence of multiple, loosely coupled partners using digital tools for collaboration. We show what the digital tools afford using the concept of sociability. We discuss how distributed digital practices make teams more task-oriented and that creating and maintaining sociability, a key issue for knowledge sharing in agile ISD organizations, require relation oriented communication during practical problem solving using digital tools.
△ Less
Submitted 10 November, 2021;
originally announced November 2021.
-
Radio Galaxy Zoo: Unsupervised Clustering of Convolutionally Auto-encoded Radio-astronomical Images
Authors:
Nicholas O. Ralph,
Ray P. Norris,
Gu Fang,
Laurence A. F. Park,
Timothy J. Galvin,
Matthew J. Alger,
Heinz Andernach,
Chris Lintott,
Lawrence Rudnick,
Stanislav Shabala,
O. Ivy Wong
Abstract:
This paper demonstrates a novel and efficient unsupervised clustering method with the combination of a Self-Organising Map (SOM) and a convolutional autoencoder. The rapidly increasing volume of radio-astronomical data has increased demand for machine learning methods as solutions to classification and outlier detection. Major astronomical discoveries are unplanned and found in the unexpected, mak…
▽ More
This paper demonstrates a novel and efficient unsupervised clustering method with the combination of a Self-Organising Map (SOM) and a convolutional autoencoder. The rapidly increasing volume of radio-astronomical data has increased demand for machine learning methods as solutions to classification and outlier detection. Major astronomical discoveries are unplanned and found in the unexpected, making unsupervised machine learning highly desirable by operating without assumptions and labelled training data. Our approach shows SOM training time is drastically reduced and high-level features can be clustered by training on auto-encoded feature vectors instead of raw images. Our results demonstrate this method is capable of accurately separating outliers on a SOM with neighbourhood similarity and K-means clustering of radio-astronomical features complexity. We present this method as a powerful new approach to data exploration by providing a detailed understanding of the morphology and relationships of Radio Galaxy Zoo (RGZ) dataset image features which can be applied to new radio survey data.
△ Less
Submitted 6 June, 2019;
originally announced June 2019.
-
The World's First Real-Time Testbed for Massive MIMO: Design, Implementation, and Validation
Authors:
Steffen Malkowsky,
Joao Vieira,
Liang Liu,
Paul Harris,
Karl Nieman,
Nikhil Kundargi,
Ian Wong,
Fredrik Tufvesson,
Viktor Öwall,
Ove Edfors
Abstract:
This paper sets up a framework for designing a massive multiple-input multiple-output (MIMO) testbed by investigating hardware (HW) and system-level requirements such as processing complexity, duplexing mode and frame structure. Taking these into account, a generic system and processing partitioning is proposed which allows flexible scaling and processing distribution onto a multitude of physicall…
▽ More
This paper sets up a framework for designing a massive multiple-input multiple-output (MIMO) testbed by investigating hardware (HW) and system-level requirements such as processing complexity, duplexing mode and frame structure. Taking these into account, a generic system and processing partitioning is proposed which allows flexible scaling and processing distribution onto a multitude of physically separated devices. Based on the given HW constraints such as maximum number of links and maximum throughput for peer-to-peer interconnections combined with processing capabilities, the framework allows to evaluate modular HW components. To verify our design approach, we present the LuMaMi (Lund University Massive MIMO) testbed which constitutes the first reconfigurable real-time HW platform for prototy** massive MIMO. Utilizing up to 100 base station antennas and more than 50 Field Programmable Gate Arrays, up to 12 user equipments are served on the same time/frequency resource using an LTE-like Orthogonal Frequency Division Multiplexing time-division duplex-based transmission scheme. Proof-of-concept tests with this system show that massive MIMO can simultaneously serve a multitude of users in a static indoor and static outdoor environment utilizing the same time/frequency resource.
△ Less
Submitted 16 May, 2017; v1 submitted 20 December, 2016;
originally announced January 2017.
-
Creating Interactive Behaviors in Early Sketch by Recording and Remixing Crowd Demonstrations
Authors:
Sang Won Lee,
Yi Wei Yang,
Shiyan Yan,
Yu** Zhang,
Isabelle Wong,
Zhengxi Tan,
Miles McGruder,
Christopher Homan,
Walter Lasecki
Abstract:
In the early stages of designing graphical user interfaces (GUIs), the look (appearance) can be easily presented by sketching, but the feel (interactive behaviors) cannot, and often requires an accompanying description of how it works (Myers et al. 2008). We propose to use crowdsourcing to augment early sketches with interactive behaviors generated, used, and reused by collective "wizards-of-oz" a…
▽ More
In the early stages of designing graphical user interfaces (GUIs), the look (appearance) can be easily presented by sketching, but the feel (interactive behaviors) cannot, and often requires an accompanying description of how it works (Myers et al. 2008). We propose to use crowdsourcing to augment early sketches with interactive behaviors generated, used, and reused by collective "wizards-of-oz" as opposed to a single wizard as in prior work (Davis et al. 2007). This demo presents an extension of Apparition (Lasecki et al. 2015), a crowd-powered prototy** tool that allows end users to create functional GUIs using speech and sketch. In Apparition, crowd workers collaborate in real-time on a shared canvas to refine the user-requested sketch interactively, and with the assistance of the end users. Our demo extends this functionality to let crowd workers "demonstrate" the canvas changes that are needed for a behavior and refine their demonstrations to improve the fidelity of interactive behaviors. The system then lets workers "remix" these behaviors to make creating future behaviors more efficient.
△ Less
Submitted 5 September, 2016;
originally announced September 2016.
-
Design and Implementation of a TDD-Based 128-Antenna Massive MIMO Prototy** System
Authors:
Xi Yang,
Wen-Jun Lu,
Ning Wang,
Karl Nieman,
Shi **,
Hongbo Zhu,
Xiaomin Mu,
Ian Wong,
Yongming Huang,
Xiaohu You
Abstract:
Spurred by the dramatic mobile IP growth and the emerging Internet of Things (IoT) and cloud-based applications, wireless networking is witnessing a paradigm shift. By fully exploiting the spatial degrees of freedom, the massive multipleinput- multiple-output (MIMO) technology promises significant gains in both data rates and link reliability. This paper presents a time-division duplex (TDD)-based…
▽ More
Spurred by the dramatic mobile IP growth and the emerging Internet of Things (IoT) and cloud-based applications, wireless networking is witnessing a paradigm shift. By fully exploiting the spatial degrees of freedom, the massive multipleinput- multiple-output (MIMO) technology promises significant gains in both data rates and link reliability. This paper presents a time-division duplex (TDD)-based 128-antenna massive MIMO prototy** system designed to operate on a 20 MHz bandwidth. Up to twelve single-antenna users can be served by the designed system at the same time. System model is provided and link-level simulation corresponding to our practical TDDbased massive MIMO prototy** system is conducted to validate our design and performance of the algorithms. Based on the system hardware design demonstrated in this paper, both uplink real-time video and downlink data transmissions are realized, and the experiment results show that 268.8 Mbps rate was achieved for eight single-antenna users using QPSK modulation. The maximum spectral efficiency of the designed system will be 80.64 bit/s/Hz by twelve single-antenna users with 256-QAM modulation.
△ Less
Submitted 26 August, 2016;
originally announced August 2016.
-
Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation
Authors:
Dominique Guillot,
Bala Rajaratnam,
Benjamin T. Rolfs,
Arian Maleki,
Ian Wong
Abstract:
The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized covariance matrix estimation is presented. Although numerous algorithms have been…
▽ More
The L1-regularized maximum likelihood estimation problem has recently become a topic of great interest within the machine learning, statistics, and optimization communities as a method for producing sparse inverse covariance estimators. In this paper, a proximal gradient method (G-ISTA) for performing L1-regularized covariance matrix estimation is presented. Although numerous algorithms have been proposed for solving this problem, this simple proximal gradient method is found to have attractive theoretical and numerical properties. G-ISTA has a linear rate of convergence, resulting in an O(log e) iteration complexity to reach a tolerance of e. This paper gives eigenvalue bounds for the G-ISTA iterates, providing a closed-form linear convergence rate. The rate is shown to be closely related to the condition number of the optimal point. Numerical convergence results and timing comparisons for the proposed method are presented. G-ISTA is shown to perform very well, especially when the optimal point is well-conditioned.
△ Less
Submitted 26 November, 2012; v1 submitted 12 November, 2012;
originally announced November 2012.
