Skip to main content

Showing 1–16 of 16 results for author: Nevado-Holgado, A

.
  1. arXiv:2403.19802  [pdf, other

    cs.CL cs.AI

    Develo** Healthcare Language Model Embedding Spaces

    Authors: Niall Taylor, Dan Schofield, Andrey Kormilitzin, Dan W Joyce, Alejo Nevado-Holgado

    Abstract: Pre-trained Large Language Models (LLMs) often struggle on out-of-domain datasets like healthcare focused text. We explore specialized pre-training to adapt smaller LLMs to different healthcare datasets. Three methods are assessed: traditional masked language modeling, Deep Contrastive Learning for Unsupervised Textual Representations (DeCLUTR), and a novel pre-training objective utilizing metadat… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

  2. arXiv:2403.19790  [pdf, other

    cs.AI

    Bespoke Large Language Models for Digital Triage Assistance in Mental Health Care

    Authors: Niall Taylor, Andrey Kormilitzin, Isabelle Lorge, Alejo Nevado-Holgado, Dan W Joyce

    Abstract: Contemporary large language models (LLMs) may have utility for processing unstructured, narrative free-text clinical data contained in electronic health records (EHRs) -- a particularly important use-case for mental health where a majority of routinely-collected patient data lacks structured, machine-readable content. A significant problem for the the United Kingdom's National Health Service (NH… ▽ More

    Submitted 28 March, 2024; originally announced March 2024.

  3. arXiv:2402.10597  [pdf, other

    cs.CL cs.AI

    Efficiency at Scale: Investigating the Performance of Diminutive Language Models in Clinical Tasks

    Authors: Niall Taylor, Upamanyu Ghose, Omid Rohanian, Mohammadmahdi Nouriborji, Andrey Kormilitzin, David Clifton, Alejo Nevado-Holgado

    Abstract: The entry of large language models (LLMs) into research and commercial spaces has led to a trend of ever-larger models, with initial promises of generalisability, followed by a widespread desire to downsize and create specialised models without the need for complete fine-tuning, using Parameter Efficient Fine-tuning (PEFT) methods. We present an investigation into the suitability of different PEFT… ▽ More

    Submitted 16 February, 2024; originally announced February 2024.

  4. arXiv:2402.07645  [pdf, other

    cs.CL

    Detecting the Clinical Features of Difficult-to-Treat Depression using Synthetic Data from Large Language Models

    Authors: Isabelle Lorge, Dan W. Joyce, Niall Taylor, Alejo Nevado-Holgado, Andrea Cipriani, Andrey Kormilitzin

    Abstract: Difficult-to-treat depression (DTD) has been proposed as a broader and more clinically comprehensive perspective on a person's depressive disorder where despite treatment, they continue to experience significant burden. We sought to develop a Large Language Model (LLM)-based tool capable of interrogating routinely-collected, narrative (free-text) electronic health record (EHR) data to locate publi… ▽ More

    Submitted 12 February, 2024; originally announced February 2024.

  5. arXiv:2311.12884  [pdf, other

    q-bio.GN cs.LG q-bio.QM

    Identifying DNA Sequence Motifs Using Deep Learning

    Authors: Asmita Poddar, Vladimir Uzun, Elizabeth Tunbridge, Wilfried Haerty, Alejo Nevado-Holgado

    Abstract: Splice sites play a crucial role in gene expression, and accurate prediction of these sites in DNA sequences is essential for diagnosing and treating genetic disorders. We address the challenge of splice site prediction by introducing DeepDeCode, an attention-based deep learning sequence model to capture the long-term dependencies in the nucleotides in DNA sequences. We further propose using visua… ▽ More

    Submitted 20 November, 2023; originally announced November 2023.

  6. arXiv:2208.00033  [pdf, other

    cs.LG cs.AI

    Personalised recommendations of sleep behaviour with neural networks using sleep diaries captured in Sleepio

    Authors: Alejo Nevado-Holgado, Colin Espie, Maria Liakata, Alasdair Henry, Jenny Gu, Niall Taylor, Kate Saunders, Tom Walker, Chris Miller

    Abstract: SleepioTM is a digital mobile phone and web platform that uses techniques from cognitive behavioural therapy (CBT) to improve sleep in people with sleep difficulty. As part of this process, Sleepio captures data about the sleep behaviour of the users that have consented to such data being processed. For neural networks, the scale of the data is an opportunity to train meaningful models translatabl… ▽ More

    Submitted 29 July, 2022; originally announced August 2022.

  7. arXiv:2205.05535  [pdf, other

    cs.CL

    Clinical Prompt Learning with Frozen Language Models

    Authors: Niall Taylor, Yi Zhang, Dan Joyce, Alejo Nevado-Holgado, Andrey Kormilitzin

    Abstract: Prompt learning is a new paradigm in the Natural Language Processing (NLP) field which has shown impressive performance on a number of natural language tasks with common benchmarking text datasets in full, few-shot, and zero-shot train-evaluation setups. Recently, it has even been observed that large but frozen pre-trained language models (PLMs) with prompt learning outperform smaller but fine-tun… ▽ More

    Submitted 11 May, 2022; originally announced May 2022.

    Comments: 18 pages, 6 figures, 6 tables

    MSC Class: ACM-class: J.2

  8. arXiv:2111.07611  [pdf, other

    cs.CL cs.AI

    Rationale production to support clinical decision-making

    Authors: Niall Taylor, Lei Sha, Dan W Joyce, Thomas Lukasiewicz, Alejo Nevado-Holgado, Andrey Kormilitzin

    Abstract: The development of neural networks for clinical artificial intelligence (AI) is reliant on interpretability, transparency, and performance. The need to delve into the black-box neural network and derive interpretable explanations of model output is paramount. A task of high clinical importance is predicting the likelihood of a patient being readmitted to hospital in the near future to enable effic… ▽ More

    Submitted 15 November, 2021; originally announced November 2021.

    Comments: Machine Learning for Health (ML4H) - Extended Abstract

  9. arXiv:2010.12260  [pdf, other

    cs.LG cs.CV stat.ML

    Population Gradients improve performance across data-sets and architectures in object classification

    Authors: Yurika Sakai, Andrey Kormilitzin, Qiang Liu, Alejo Nevado-Holgado

    Abstract: The most successful methods such as ReLU transfer functions, batch normalization, Xavier initialization, dropout, learning rate decay, or dynamic optimizers, have become standards in the field due, particularly, to their ability to increase the performance of Neural Networks (NNs) significantly and in almost all situations. Here we present a new method to calculate the gradients while training NNs… ▽ More

    Submitted 23 October, 2020; originally announced October 2020.

  10. arXiv:2010.08433  [pdf, other

    cs.CL cs.IR

    An efficient representation of chronological events in medical texts

    Authors: Andrey Kormilitzin, Nemanja Vaci, Qiang Liu, Hao Ni, Goran Nenadic, Alejo Nevado-Holgado

    Abstract: In this work we addressed the problem of capturing sequential information contained in longitudinal electronic health records (EHRs). Clinical notes, which is a particular type of EHR data, are a rich source of information and practitioners often develop clever solutions how to maximise the sequential information contained in free-texts. We proposed a systematic methodology for learning from chron… ▽ More

    Submitted 24 October, 2020; v1 submitted 16 October, 2020; originally announced October 2020.

    Comments: 4 pages, 2 figures, 7 tables

  11. arXiv:2010.04897  [pdf, other

    cs.CL

    Information Extraction from Swedish Medical Prescriptions with Sig-Transformer Encoder

    Authors: John Pougue Biyong, Bo Wang, Terry Lyons, Alejo J Nevado-Holgado

    Abstract: Relying on large pretrained language models such as Bidirectional Encoder Representations from Transformers (BERT) for encoding and adding a simple prediction layer has led to impressive performance in many clinical natural language processing (NLP) tasks. In this work, we present a novel extension to the Transformer architecture, by incorporating signature transform with the self-attention model.… ▽ More

    Submitted 10 October, 2020; originally announced October 2020.

    MSC Class: 60L10 ACM Class: I.2.7

  12. arXiv:2008.03408  [pdf, other

    cs.LG cs.CL eess.AS stat.ML

    Learning to Detect Bipolar Disorder and Borderline Personality Disorder with Language and Speech in Non-Clinical Interviews

    Authors: Bo Wang, Yue Wu, Niall Taylor, Terry Lyons, Maria Liakata, Alejo J Nevado-Holgado, Kate E A Saunders

    Abstract: Bipolar disorder (BD) and borderline personality disorder (BPD) are both chronic psychiatric disorders. However, their overlap** symptoms and common comorbidity make it challenging for the clinicians to distinguish the two conditions on the basis of a clinical interview. In this work, we first present a new multi-modal dataset containing interviews involving individuals with BD or BPD being inte… ▽ More

    Submitted 31 May, 2021; v1 submitted 7 August, 2020; originally announced August 2020.

    MSC Class: 60L10

  13. arXiv:2003.01271  [pdf, other

    cs.CL cs.IR cs.LG

    Med7: a transferable clinical natural language processing model for electronic health records

    Authors: Andrey Kormilitzin, Nemanja Vaci, Qiang Liu, Alejo Nevado-Holgado

    Abstract: The field of clinical natural language processing has been advanced significantly since the introduction of deep learning models. The self-supervised representation learning and the transfer learning paradigm became the methods of choice in many natural language processing application, in particular in the settings with the dearth of high quality manually annotated data. Electronic health record s… ▽ More

    Submitted 24 April, 2020; v1 submitted 2 March, 2020; originally announced March 2020.

    Comments: 16 pages, 1 figure, 15 tables

  14. arXiv:1908.11399  [pdf, other

    eess.IV cs.LG q-bio.QM stat.ML

    Deep Learning for Estimating Synaptic Health of Primary Neuronal Cell Culture

    Authors: Andrey Kormilitzin, Xinyu Yang, William H. Stone, Caroline Woffindale, Francesca Nicholls, Elena Ribe, Alejo Nevado-Holgado, Noel Buckley

    Abstract: Understanding the morphological changes of primary neuronal cells induced by chemical compounds is essential for drug discovery. Using the data from a single high-throughput imaging assay, a classification model for predicting the biological activity of candidate compounds was introduced. The image recognition model which is based on deep convolutional neural network (CNN) architecture with residu… ▽ More

    Submitted 29 August, 2019; originally announced August 2019.

    Comments: 11 pages, 5 figures

  15. arXiv:1901.01592  [pdf

    cs.CL cs.AI stat.ML

    Named Entity Recognition in Electronic Health Records Using Transfer Learning Bootstrapped Neural Networks

    Authors: Luka Gligic, Andrey Kormilitzin, Paul Goldberg, Alejo Nevado-Holgado

    Abstract: Neural networks (NNs) have become the state of the art in many machine learning applications, especially in image and sound processing [1]. The same, although to a lesser extent [2,3], could be said in natural language processing (NLP) tasks, such as named entity recognition. However, the success of NNs remains dependent on the availability of large labelled datasets, which is a significant hurdle… ▽ More

    Submitted 29 July, 2019; v1 submitted 6 January, 2019; originally announced January 2019.

    Comments: 11 pages, 4 figures, 8 tables

  16. arXiv:1811.05468  [pdf

    cs.CL cs.LG stat.ML

    Few-shot Learning for Named Entity Recognition in Medical Text

    Authors: Maximilian Hofer, Andrey Kormilitzin, Paul Goldberg, Alejo Nevado-Holgado

    Abstract: Deep neural network models have recently achieved state-of-the-art performance gains in a variety of natural language processing (NLP) tasks (Young, Hazarika, Poria, & Cambria, 2017). However, these gains rely on the availability of large amounts of annotated examples, without which state-of-the-art performance is rarely achievable. This is especially inconvenient for the many NLP fields where ann… ▽ More

    Submitted 13 November, 2018; originally announced November 2018.

    Comments: 10 pages, 4 figures, 4 tables