SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 13 Languages
Authors:
Nedjma Ousidhoum,
Shamsuddeen Hassan Muhammad,
Mohamed Abdalla,
Idris Abdulmumin,
Ibrahim Said Ahmad,
Sanchit Ahuja,
Alham Fikri Aji,
Vladimir Araujo,
Abinew Ali Ayele,
Pavan Baswani,
Meriem Beloucif,
Chris Biemann,
Sofia Bourhim,
Christine De Kock,
Genet Shanko Dekebo,
Oumaima Hourrane,
Gopichand Kanumolu,
Lokesh Madasu,
Samuel Rutunda,
Manish Shrivastava,
Thamar Solorio,
Nirmal Surange,
Hailegnaw Getaneh Tilaye,
Krishnapriya Vishnubhotla,
Genta Winata
, et al. (2 additional authors not shown)
Abstract:
Exploring and quantifying semantic relatedness is central to representing language and holds significant implications across various NLP tasks. While earlier NLP research primarily focused on semantic similarity, often within the English language context, we instead investigate the broader phenomenon of semantic relatedness. In this paper, we present \textit{SemRel}, a new semantic relatedness dat…
▽ More
Exploring and quantifying semantic relatedness is central to representing language and holds significant implications across various NLP tasks. While earlier NLP research primarily focused on semantic similarity, often within the English language context, we instead investigate the broader phenomenon of semantic relatedness. In this paper, we present \textit{SemRel}, a new semantic relatedness dataset collection annotated by native speakers across 13 languages: \textit{Afrikaans, Algerian Arabic, Amharic, English, Hausa, Hindi, Indonesian, Kinyarwanda, Marathi, Moroccan Arabic, Modern Standard Arabic, Spanish,} and \textit{Telugu}. These languages originate from five distinct language families and are predominantly spoken in Africa and Asia -- regions characterised by a relatively limited availability of NLP resources. Each instance in the SemRel datasets is a sentence pair associated with a score that represents the degree of semantic textual relatedness between the two sentences. The scores are obtained using a comparative annotation framework. We describe the data collection and annotation processes, challenges when building the datasets, baseline experiments, and their impact and utility in NLP.
△ Less
Submitted 31 May, 2024; v1 submitted 13 February, 2024;
originally announced February 2024.
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages
Authors:
Shamsuddeen Hassan Muhammad,
Idris Abdulmumin,
Abinew Ali Ayele,
Nedjma Ousidhoum,
David Ifeoluwa Adelani,
Seid Muhie Yimam,
Ibrahim Sa'id Ahmad,
Meriem Beloucif,
Saif M. Mohammad,
Sebastian Ruder,
Oumaima Hourrane,
Pavel Brazdil,
Felermino Dário Mário António Ali,
Davis David,
Salomey Osei,
Bello Shehu Bello,
Falalu Ibrahim,
Tajuddeen Gwadabe,
Samuel Rutunda,
Tadesse Belay,
Wendimu Baye Messelle,
Hailu Beshada Balcha,
Sisay Adugna Chala,
Hagos Tesfahun Gebremichael,
Bernard Opoku
, et al. (1 additional authors not shown)
Abstract:
Africa is home to over 2,000 languages from more than six language families and has the highest linguistic diversity among all continents. These include 75 languages with at least one million speakers each. Yet, there is little NLP research conducted on African languages. Crucial to enabling such research is the availability of high-quality annotated datasets. In this paper, we introduce AfriSenti…
▽ More
Africa is home to over 2,000 languages from more than six language families and has the highest linguistic diversity among all continents. These include 75 languages with at least one million speakers each. Yet, there is little NLP research conducted on African languages. Crucial to enabling such research is the availability of high-quality annotated datasets. In this paper, we introduce AfriSenti, a sentiment analysis benchmark that contains a total of >110,000 tweets in 14 African languages (Amharic, Algerian Arabic, Hausa, Igbo, Kinyarwanda, Moroccan Arabic, Mozambican Portuguese, Nigerian Pidgin, Oromo, Swahili, Tigrinya, Twi, Xitsonga, and Yorùbá) from four language families. The tweets were annotated by native speakers and used in the AfriSenti-SemEval shared task (The AfriSenti Shared Task had over 200 participants. See website at https://afrisenti-semeval.github.io). We describe the data collection methodology, annotation process, and the challenges we dealt with when curating each dataset. We further report baseline experiments conducted on the different datasets and discuss their usefulness.
△ Less
Submitted 4 November, 2023; v1 submitted 17 February, 2023;
originally announced February 2023.
