Walia-LLM: Enhancing Amharic-LLaMA by Integrating Task-Specific and Generative Datasets
Authors:
Israel Abebe Azime,
Atnafu Lambebo Tonja,
Tadesse Destaw Belay,
Mitiku Yohannes Fuge,
Aman Kassahun Wassie,
Eyasu Shiferaw Jada,
Yonas Chanie,
Walelign Tewabe Sewunetie,
Seid Muhie Yimam
Abstract:
Large language models (LLMs) have received a lot of attention in natural language processing (NLP) research because of their exceptional performance in understanding and generating human languages. However, low-resource languages are left behind due to the unavailability of resources. In this work, we focus on enhancing the LLaMA-2-Amharic model by integrating task-specific and generative datasets…
▽ More
Large language models (LLMs) have received a lot of attention in natural language processing (NLP) research because of their exceptional performance in understanding and generating human languages. However, low-resource languages are left behind due to the unavailability of resources. In this work, we focus on enhancing the LLaMA-2-Amharic model by integrating task-specific and generative datasets to improve language model performance for Amharic. We compile an Amharic instruction fine-tuning dataset and fine-tuned LLaMA-2-Amharic model. The fine-tuned model shows promising results in different NLP tasks. We open-source our dataset creation pipeline, instruction datasets, trained models, and evaluation outputs to promote language-specific studies on these models.
△ Less
Submitted 29 April, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
