-
Explanatory Argument Extraction of Correct Answers in Resident Medical Exams
Authors:
Iakes Goenaga,
Aitziber Atutxa,
Koldo Gojenola,
Maite Oronoz,
Rodrigo Agerri
Abstract:
Develo** the required technology to assist medical experts in their everyday activities is currently a hot topic in the Artificial Intelligence research field. Thus, a number of large language models (LLMs) and automated benchmarks have recently been proposed with the aim of facilitating information extraction in Evidence-Based Medicine (EBM) using natural language as a tool for mediating in hum…
▽ More
Develo** the required technology to assist medical experts in their everyday activities is currently a hot topic in the Artificial Intelligence research field. Thus, a number of large language models (LLMs) and automated benchmarks have recently been proposed with the aim of facilitating information extraction in Evidence-Based Medicine (EBM) using natural language as a tool for mediating in human-AI interaction. The most representative benchmarks are limited to either multiple-choice or long-form answers and are available only in English. In order to address these shortcomings, in this paper we present a new dataset which, unlike previous work: (i) includes not only explanatory arguments for the correct answer, but also arguments to reason why the incorrect answers are not correct; (ii) the explanations are written originally by medical doctors to answer questions from the Spanish Residency Medical Exams. Furthermore, this new benchmark allows us to setup a novel extractive task which consists of identifying the explanation of the correct answer written by medical doctors. An additional benefit of our setting is that we can leverage the extractive QA paradigm to automatically evaluate performance of LLMs without resorting to costly manual evaluation by medical experts. Comprehensive experimentation with language models for Spanish shows that sometimes multilingual models fare better than monolingual ones, even outperforming models which have been adapted to the medical domain. Furthermore, results across the monolingual models are mixed, with supposedly smaller and inferior models performing competitively. In any case, the obtained results show that our novel dataset and approach can be an effective technique to help medical practitioners in identifying relevant evidence-based explanations for medical questions.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
-
HiTZ@Antidote: Argumentation-driven Explainable Artificial Intelligence for Digital Medicine
Authors:
Rodrigo Agerri,
Iñigo Alonso,
Aitziber Atutxa,
Ander Berrondo,
Ainara Estarrona,
Iker Garcia-Ferrero,
Iakes Goenaga,
Koldo Gojenola,
Maite Oronoz,
Igor Perez-Tejedor,
German Rigau,
Anar Yeginbergenova
Abstract:
Providing high quality explanations for AI predictions based on machine learning is a challenging and complex task. To work well it requires, among other factors: selecting a proper level of generality/specificity of the explanation; considering assumptions about the familiarity of the explanation beneficiary with the AI task under consideration; referring to specific elements that have contribute…
▽ More
Providing high quality explanations for AI predictions based on machine learning is a challenging and complex task. To work well it requires, among other factors: selecting a proper level of generality/specificity of the explanation; considering assumptions about the familiarity of the explanation beneficiary with the AI task under consideration; referring to specific elements that have contributed to the decision; making use of additional knowledge (e.g. expert evidence) which might not be part of the prediction process; and providing evidence supporting negative hypothesis. Finally, the system needs to formulate the explanation in a clearly interpretable, and possibly convincing, way. Given these considerations, ANTIDOTE fosters an integrated vision of explainable AI, where low-level characteristics of the deep learning process are combined with higher level schemes proper of the human argumentation capacity. ANTIDOTE will exploit cross-disciplinary competences in deep learning and argumentation to support a broader and innovative view of explainable AI, where the need for high-quality explanations for clinical cases deliberation is critical. As a first result of the project, we publish the Antidote CasiMedicos dataset to facilitate research on explainable AI in general, and argumentation in the medical domain in particular.
△ Less
Submitted 9 June, 2023;
originally announced June 2023.
-
Time Synchronization in modular collaborative robots
Authors:
Carlos San Vicente Gutiérrez,
Lander Usategui San Juan,
Irati Zamalloa Ugarte,
Iñigo Muguruza Goenaga,
Laura Alzola Kirschgens,
Víctor Mayoral Vilches
Abstract:
A new generation of robot systems which are modular, flexible and safe for human-robot interaction are needed. Existing cobots seem to meet only the later and require a modular approach to improve their reconfigurability and interoperability. We propose a new sub-class of cobots named M-cobots which tackle these problems. In particular, we discuss the relevance of synchronization for these systems…
▽ More
A new generation of robot systems which are modular, flexible and safe for human-robot interaction are needed. Existing cobots seem to meet only the later and require a modular approach to improve their reconfigurability and interoperability. We propose a new sub-class of cobots named M-cobots which tackle these problems. In particular, we discuss the relevance of synchronization for these systems, analyze it and demonstrate how with a properly configured M-cobot, we are able to obtain a) distributed sub-microsecond clock synchronization accuracy among modules, b) timestam** accuracy of ROS 2.0 messages under 100 microseconds and c) millisecond-level end-to-end communication latencies, even when disturbed with networking overloads of up to 90% of the network capacity.
△ Less
Submitted 19 September, 2018;
originally announced September 2018.
