Introducing Bode: A Fine-Tuned Large Language Model for Portuguese Prompt-Based Task
Authors:
Gabriel Lino Garcia,
Pedro Henrique Paiola,
Luis Henrique Morelli,
Giovani Candido,
Arnaldo Cândido Júnior,
Danilo Samuel Jodas,
Luis C. S. Afonso,
Ivan Rizzo Guilherme,
Bruno Elias Penteado,
João Paulo Papa
Abstract:
Large Language Models (LLMs) are increasingly bringing advances to Natural Language Processing. However, low-resource languages, those lacking extensive prominence in datasets for various NLP tasks, or where existing datasets are not as substantial, such as Portuguese, already obtain several benefits from LLMs, but not to the same extent. LLMs trained on multilingual datasets normally struggle to…
▽ More
Large Language Models (LLMs) are increasingly bringing advances to Natural Language Processing. However, low-resource languages, those lacking extensive prominence in datasets for various NLP tasks, or where existing datasets are not as substantial, such as Portuguese, already obtain several benefits from LLMs, but not to the same extent. LLMs trained on multilingual datasets normally struggle to respond to prompts in Portuguese satisfactorily, presenting, for example, code switching in their responses. This work proposes a fine-tuned LLaMA 2-based model for Portuguese prompts named Bode in two versions: 7B and 13B. We evaluate the performance of this model in classification tasks using the zero-shot approach with in-context learning, and compare it with other LLMs. Our main contribution is to bring an LLM with satisfactory results in the Portuguese language, as well as to provide a model that is free for research or commercial purposes.
△ Less
Submitted 5 January, 2024;
originally announced January 2024.
Evaluating Wikipedia as a source of information for disease understanding
Authors:
Eduardo P. Garcia del Valle,
Gerardo Lagunes Garcia,
Lucia Prieto Santamaria,
Massimiliano Zanin,
Alejandro Rodriguez-Gonzalez,
Ernestina Menasalvas Ruiz
Abstract:
The increasing availability of biological data is improving our understanding of diseases and providing new insight into their underlying relationships. Thanks to the improvements on both text mining techniques and computational capacity, the combination of biological data with semantic information obtained from medical publications has proven to be a very promising path. However, the limitations…
▽ More
The increasing availability of biological data is improving our understanding of diseases and providing new insight into their underlying relationships. Thanks to the improvements on both text mining techniques and computational capacity, the combination of biological data with semantic information obtained from medical publications has proven to be a very promising path. However, the limitations in the access to these data and their lack of structure pose challenges to this approach. In this document we propose the use of Wikipedia - the free online encyclopedia - as a source of accessible textual information for disease understanding research. To check its validity, we compare its performance in the determination of relationships between diseases with that of PubMed, one of the most consulted data sources of medical texts. The obtained results suggest that the information extracted from Wikipedia is as relevant as that obtained from PubMed abstracts (i.e. the free access portion of its articles), although further research is proposed to verify its reliability for medical studies.
△ Less
Submitted 4 August, 2018;
originally announced August 2018.
