The African Stopwords project: curating stopwords for African languages
Authors:
Chris Emezue,
Hellina Nigatu,
Cynthia Thinwa,
Helper Zhou,
Shamsuddeen Muhammad,
Lerato Louis,
Idris Abdulmumin,
Samuel Oyerinde,
Benjamin Ajibade,
Olanrewaju Samuel,
Oviawe Joshua,
Emeka Onwuegbuzia,
Handel Emezue,
Ifeoluwatayo A. Ige,
Atnafu Lambebo Tonja,
Chiamaka Chukwuneke,
Bonaventure F. P. Dossou,
Naome A. Etori,
Mbonu Chinedu Emmanuel,
Oreen Yousuf,
Kaosarat Aina,
Davis David
Abstract:
Stopwords are fundamental in Natural Language Processing (NLP) techniques for information retrieval. One of the common tasks in preprocessing of text data is the removal of stopwords. Currently, while high-resource languages like English benefit from the availability of several stopwords, low-resource languages, such as those found in the African continent, have none that are standardized and avai…
▽ More
Stopwords are fundamental in Natural Language Processing (NLP) techniques for information retrieval. One of the common tasks in preprocessing of text data is the removal of stopwords. Currently, while high-resource languages like English benefit from the availability of several stopwords, low-resource languages, such as those found in the African continent, have none that are standardized and available for use in NLP packages. Stopwords in the context of African languages are understudied and can reveal information about the crossover between languages. The \textit{African Stopwords} project aims to study and curate stopwords for African languages. In this paper, we present our current progress on ten African languages as well as future plans for the project.
△ Less
Submitted 21 March, 2023;
originally announced April 2023.
MasakhaNER: Named Entity Recognition for African Languages
Authors:
David Ifeoluwa Adelani,
Jade Abbott,
Graham Neubig,
Daniel D'souza,
Julia Kreutzer,
Constantine Lignos,
Chester Palen-Michel,
Happy Buzaaba,
Shruti Rijhwani,
Sebastian Ruder,
Stephen Mayhew,
Israel Abebe Azime,
Shamsuddeen Muhammad,
Chris Chinenye Emezue,
Joyce Nakatumba-Nabende,
Perez Ogayo,
Anuoluwapo Aremu,
Catherine Gitau,
Derguene Mbaye,
Jesujoba Alabi,
Seid Muhie Yimam,
Tajuddeen Gwadabe,
Ignatius Ezeani,
Rubungo Andre Niyongabo,
Jonathan Mukiibi
, et al. (36 additional authors not shown)
Abstract:
We take a step towards addressing the under-representation of the African continent in NLP research by creating the first large publicly available high-quality dataset for named entity recognition (NER) in ten African languages, bringing together a variety of stakeholders. We detail characteristics of the languages to help researchers understand the challenges that these languages pose for NER. We…
▽ More
We take a step towards addressing the under-representation of the African continent in NLP research by creating the first large publicly available high-quality dataset for named entity recognition (NER) in ten African languages, bringing together a variety of stakeholders. We detail characteristics of the languages to help researchers understand the challenges that these languages pose for NER. We analyze our datasets and conduct an extensive empirical evaluation of state-of-the-art methods across both supervised and transfer learning settings. We release the data, code, and models in order to inspire future research on African NLP.
△ Less
Submitted 5 July, 2021; v1 submitted 22 March, 2021;
originally announced March 2021.
