Aloe: A Family of Fine-tuned Open Healthcare LLMs
Authors:
Ashwin Kumar Gururajan,
Enrique Lopez-Cuena,
Jordi Bayarri-Planas,
Adrian Tormos,
Daniel Hinjos,
Pablo Bernabeu-Perez,
Anna Arias-Duart,
Pablo Agustin Martin-Torres,
Lucia Urcelay-Ganzabal,
Marta Gonzalez-Mallo,
Sergio Alvarez-Napagao,
Eduard Ayguadé-Parra,
Ulises Cortés Dario Garcia-Gasulla
Abstract:
As the capabilities of Large Language Models (LLMs) in healthcare and medicine continue to advance, there is a growing need for competitive open-source models that can safeguard public interest. With the increasing availability of highly competitive open base models, the impact of continued pre-training is increasingly uncertain. In this work, we explore the role of instruct tuning, model merging,…
▽ More
As the capabilities of Large Language Models (LLMs) in healthcare and medicine continue to advance, there is a growing need for competitive open-source models that can safeguard public interest. With the increasing availability of highly competitive open base models, the impact of continued pre-training is increasingly uncertain. In this work, we explore the role of instruct tuning, model merging, alignment, red teaming and advanced inference schemes, as means to improve current open models. To that end, we introduce the Aloe family, a set of open medical LLMs highly competitive within its scale range. Aloe models are trained on the current best base models (Mistral, LLaMA 3), using a new custom dataset which combines public data sources improved with synthetic Chain of Thought (CoT). Aloe models undergo an alignment phase, becoming one of the first few policy-aligned open healthcare LLM using Direct Preference Optimization, setting a new standard for ethical performance in healthcare LLMs. Model evaluation expands to include various bias and toxicity datasets, a dedicated red teaming effort, and a much-needed risk assessment for healthcare LLMs. Finally, to explore the limits of current LLMs in inference, we study several advanced prompt engineering strategies to boost performance across benchmarks, yielding state-of-the-art results for open healthcare 7B LLMs, unprecedented at this scale.
△ Less
Submitted 3 May, 2024;
originally announced May 2024.
Exploring the Role of Explainability in AI-Assisted Embryo Selection
Authors:
Lucia Urcelay,
Daniel Hinjos,
Pablo A. Martin-Torres,
Marta Gonzalez,
Marta Mendez,
Salva Cívico,
Sergio Álvarez-Napagao,
Dario Garcia-Gasulla
Abstract:
In Vitro Fertilization is among the most widespread treatments for infertility. One of its main challenges is the evaluation and selection of embryo for implantation, a process with large inter- and intra-clinician variability. Deep learning based methods are gaining attention, but their opaque nature compromises their acceptance in the clinical context, where transparency in the decision making i…
▽ More
In Vitro Fertilization is among the most widespread treatments for infertility. One of its main challenges is the evaluation and selection of embryo for implantation, a process with large inter- and intra-clinician variability. Deep learning based methods are gaining attention, but their opaque nature compromises their acceptance in the clinical context, where transparency in the decision making is key. In this paper we analyze the current work in the explainability of AI-assisted embryo analysis models, identifying the limitations. We also discuss how these models could be integrated in the clinical context as decision support systems, considering the needs of clinicians and patients. Finally, we propose guidelines for the sake of increasing interpretability and trustworthiness, pushing this technology forward towards established clinical practice.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
