-
Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches
Authors:
Clément Christophe,
Praveen K Kanithi,
Prateek Munjal,
Tathagata Raha,
Nasir Hayat,
Ronnie Rajan,
Ahmed Al-Mahrooqi,
Avani Gupta,
Muhammad Umar Salman,
Gurpreet Gosal,
Bhargav Kanakiya,
Charles Chen,
Natalia Vassilieva,
Boulbaba Ben Amor,
Marco AF Pimentel,
Shadab Khan
Abstract:
This study presents a comprehensive analysis and comparison of two predominant fine-tuning methodologies - full-parameter fine-tuning and parameter-efficient tuning - within the context of medical Large Language Models (LLMs). We developed and refined a series of LLMs, based on the Llama-2 architecture, specifically designed to enhance medical knowledge retrieval, reasoning, and question-answering…
▽ More
This study presents a comprehensive analysis and comparison of two predominant fine-tuning methodologies - full-parameter fine-tuning and parameter-efficient tuning - within the context of medical Large Language Models (LLMs). We developed and refined a series of LLMs, based on the Llama-2 architecture, specifically designed to enhance medical knowledge retrieval, reasoning, and question-answering capabilities. Our experiments systematically evaluate the effectiveness of these tuning strategies across various well-known medical benchmarks. Notably, our medical LLM Med42 showed an accuracy level of 72% on the US Medical Licensing Examination (USMLE) datasets, setting a new standard in performance for openly available medical LLMs. Through this comparative analysis, we aim to identify the most effective and efficient method for fine-tuning LLMs in the medical domain, thereby contributing significantly to the advancement of AI-driven healthcare applications.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
Valveless pum** at low Reynolds numbers
Authors:
Amselem Gabriel,
Clanet Christophe,
Benzaquen Michael
Abstract:
Pum** at low Reynolds number is a ubiquitously encountered feature, both in biological organisms and engineered devices. Generating net flow requires the presence of an asymmetry in the system, which traditionally comes from geometric flow rectifiers. Here, we study a valveless system of $N$ oscillating pumps in series, where the asymmetry comes not from the geometry but from time, that is the p…
▽ More
Pum** at low Reynolds number is a ubiquitously encountered feature, both in biological organisms and engineered devices. Generating net flow requires the presence of an asymmetry in the system, which traditionally comes from geometric flow rectifiers. Here, we study a valveless system of $N$ oscillating pumps in series, where the asymmetry comes not from the geometry but from time, that is the phase shifts between the pumps. Experimental and theoretical results are in very good agreement. We provide the optimal phase shifts leading to the maximal net flow in the continuous $N\rightarrow \infty$ limit, larger by 25\% than that of a traditional peristaltic sinusoidal wave. Our results pave the way for the design of more efficient microfluidic pumps.
△ Less
Submitted 7 February, 2023; v1 submitted 11 May, 2022;
originally announced May 2022.
-
Monitoring geometrical properties of word embeddings for detecting the emergence of new topics
Authors:
Clément Christophe,
Julien Velcin,
Jairo Cugliari,
Manel Boumghar,
Philippe Suignard
Abstract:
Slow emerging topic detection is a task between event detection, where we aggregate behaviors of different words on short period of time, and language evolution, where we monitor their long term evolution. In this work, we tackle the problem of early detection of slowly emerging new topics. To this end, we gather evidence of weak signals at the word level. We propose to monitor the behavior of wor…
▽ More
Slow emerging topic detection is a task between event detection, where we aggregate behaviors of different words on short period of time, and language evolution, where we monitor their long term evolution. In this work, we tackle the problem of early detection of slowly emerging new topics. To this end, we gather evidence of weak signals at the word level. We propose to monitor the behavior of words representation in an embedding space and use one of its geometrical properties to characterize the emergence of topics. As evaluation is typically hard for this kind of task, we present a framework for quantitative evaluation. We show positive results that outperform state-of-the-art methods on two public datasets of press and scientific articles.
△ Less
Submitted 5 November, 2021;
originally announced November 2021.
-
How to detect novelty in textual data streams? A comparative study of existing methods
Authors:
Clément Christophe,
Julien Velcin,
Jairo Cugliari,
Philippe Suignard,
Manel Boumghar
Abstract:
Since datasets with annotation for novelty at the document and/or word level are not easily available, we present a simulation framework that allows us to create different textual datasets in which we control the way novelty occurs. We also present a benchmark of existing methods for novelty detection in textual data streams. We define a few tasks to solve and compare several state-of-the-art meth…
▽ More
Since datasets with annotation for novelty at the document and/or word level are not easily available, we present a simulation framework that allows us to create different textual datasets in which we control the way novelty occurs. We also present a benchmark of existing methods for novelty detection in textual data streams. We define a few tasks to solve and compare several state-of-the-art methods. The simulation framework allows us to evaluate their performances according to a set of limited scenarios and test their sensitivity to some parameters. Finally, we experiment with the same methods on different kinds of novelty in the New York Times Annotated Dataset.
△ Less
Submitted 11 September, 2019;
originally announced September 2019.
-
Study of almost everywhere convergence of series by means of martingale methods
Authors:
Cuny Christophe,
Ai Hua Fan
Abstract:
Martingale methods are used to study the almost everywhere convergence of general function series. Applications are given to ergodic series, which improves recent results of Fan \cite{FanETDS}, and to dilated series, including Davenport series, which completes results of Gaposhkin \cite{Gaposhkin67} (see also \cite{Gaposhkin68}). Application is also given to the almost everywhere convergence with…
▽ More
Martingale methods are used to study the almost everywhere convergence of general function series. Applications are given to ergodic series, which improves recent results of Fan \cite{FanETDS}, and to dilated series, including Davenport series, which completes results of Gaposhkin \cite{Gaposhkin67} (see also \cite{Gaposhkin68}). Application is also given to the almost everywhere convergence with respect to Riesz products of lacunary series.
△ Less
Submitted 27 November, 2015;
originally announced November 2015.
-
Random action of compact Lie groups and minimax estimation of a mean pattern
Authors:
Jérémie Bigot,
Claire Christophe,
Sebastien Gadat
Abstract:
This paper considers the problem of estimating a mean pattern in the setting of Grenander's pattern theory. Shape variability in a data set of curves or images is modeled by the random action of elements in a compact Lie group on an infinite dimensional space. In the case of observations contaminated by an additive Gaussian white noise, it is shown that estimating a reference template in the setti…
▽ More
This paper considers the problem of estimating a mean pattern in the setting of Grenander's pattern theory. Shape variability in a data set of curves or images is modeled by the random action of elements in a compact Lie group on an infinite dimensional space. In the case of observations contaminated by an additive Gaussian white noise, it is shown that estimating a reference template in the setting of Grenander's pattern theory falls into the category of deconvolution problems over Lie groups. To obtain this result, we build an estimator of a mean pattern by using Fourier deconvolution and harmonic analysis on compact Lie groups. In an asymptotic setting where the number of observed curves or images tends to infinity, we derive upper and lower bounds for the minimax quadratic risk over Sobolev balls. This rate depends on the smoothness of the density of the random Lie group elements representing shape variability in the data, which makes a connection between estimating a mean pattern and standard deconvolution problems in nonparametric statistics.
△ Less
Submitted 17 October, 2011; v1 submitted 13 October, 2011;
originally announced October 2011.
-
French Roadmap for complex Systems 2008-2009
Authors:
Paul Bourgine,
David Chavalarias,
Edith Perrier,
Frederic Amblard,
Francois Arlabosse,
Pierre Auger,
Jean-Bernard Baillon,
Olivier Barreteau,
Pierre Baudot,
Elisabeth Bouchaud,
Soufian Ben Amor,
Hugues Berry,
Cyrille Bertelle,
Marc Berthod,
Guillaume Beslon,
Giulio Biroli,
Daniel Bonamy,
Daniele Bourcier,
Nicolas Brodu,
Marc Bui,
Yves Burnod,
Bertrand Chapron,
Catherine Christophe,
Bruno Clement,
Jean-Louis Coatrieux
, et al. (56 additional authors not shown)
Abstract:
This second issue of the French Complex Systems Roadmap is the outcome of the Entretiens de Cargese 2008, an interdisciplinary brainstorming session organized over one week in 2008, jointly by RNSC, ISC-PIF and IXXI. It capitalizes on the first roadmap and gathers contributions of more than 70 scientists from major French institutions. The aim of this roadmap is to foster the coordination of the…
▽ More
This second issue of the French Complex Systems Roadmap is the outcome of the Entretiens de Cargese 2008, an interdisciplinary brainstorming session organized over one week in 2008, jointly by RNSC, ISC-PIF and IXXI. It capitalizes on the first roadmap and gathers contributions of more than 70 scientists from major French institutions. The aim of this roadmap is to foster the coordination of the complex systems community on focused topics and questions, as well as to present contributions and challenges in the complex systems sciences and complexity science to the public, political and industrial spheres.
△ Less
Submitted 13 July, 2009;
originally announced July 2009.
