-
DravidianCodeMix: Sentiment Analysis and Offensive Language Identification Dataset for Dravidian Languages in Code-Mixed Text
Authors:
Bharathi Raja Chakravarthi,
Ruba Priyadharshini,
Vigneshwaran Muralidaran,
Navya Jose,
Shardul Suryawanshi,
Elizabeth Sherly,
John P. McCrae
Abstract:
This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7,000 comments in Kannad…
▽ More
This paper describes the development of a multilingual, manually annotated dataset for three under-resourced Dravidian languages generated from social media comments. The dataset was annotated for sentiment analysis and offensive language identification for a total of more than 60,000 YouTube comments. The dataset consists of around 44,000 comments in Tamil-English, around 7,000 comments in Kannada-English, and around 20,000 comments in Malayalam-English. The data was manually annotated by volunteer annotators and has a high inter-annotator agreement in Krippendorff's alpha. The dataset contains all types of code-mixing phenomena since it comprises user-generated content from a multilingual country. We also present baseline experiments to establish benchmarks on the dataset using machine learning methods. The dataset is available on Github (https://github.com/bharathichezhiyan/DravidianCodeMix-Dataset) and Zenodo (https://zenodo.org/record/4750858\#.YJtw0SYo\_0M).
△ Less
Submitted 17 June, 2021;
originally announced June 2021.
-
A Sentiment Analysis Dataset for Code-Mixed Malayalam-English
Authors:
Bharathi Raja Chakravarthi,
Navya Jose,
Shardul Suryawanshi,
Elizabeth Sherly,
John P. McCrae
Abstract:
There is an increasing demand for sentiment analysis of text from social media which are mostly code-mixed. Systems trained on monolingual data fail for code-mixed data due to the complexity of mixing at different levels of the text. However, very few resources are available for code-mixed data to create models specific for this data. Although much research in multilingual and cross-lingual sentim…
▽ More
There is an increasing demand for sentiment analysis of text from social media which are mostly code-mixed. Systems trained on monolingual data fail for code-mixed data due to the complexity of mixing at different levels of the text. However, very few resources are available for code-mixed data to create models specific for this data. Although much research in multilingual and cross-lingual sentiment analysis has used semi-supervised or unsupervised methods, supervised methods still performs better. Only a few datasets for popular languages such as English-Spanish, English-Hindi, and English-Chinese are available. There are no resources available for Malayalam-English code-mixed data. This paper presents a new gold standard corpus for sentiment analysis of code-mixed text in Malayalam-English annotated by voluntary annotators. This gold standard corpus obtained a Krippendorff's alpha above 0.8 for the dataset. We use this new corpus to provide the benchmark for sentiment analysis in Malayalam-English code-mixed texts.
△ Less
Submitted 30 May, 2020;
originally announced June 2020.
-
Toward irreversibility with a finite bath of oscillators
Authors:
Artur Nogueira de São José,
Patrícia Mascarenhas Dias,
Arthur Rodrigo Bosco de Magalhães,
José Geraldo Peixoto de Faria
Abstract:
We investigate the routes by which a bath composed of a finite number of oscillators at zero temperature approaches the induction of dissipation when it nears the usual limit of dense spectrum spread in an infinite interval. It is shown that, when this limit is taken, different distributions of environment frequencies can lead to the same irreversible evolution. However, when we move away from it,…
▽ More
We investigate the routes by which a bath composed of a finite number of oscillators at zero temperature approaches the induction of dissipation when it nears the usual limit of dense spectrum spread in an infinite interval. It is shown that, when this limit is taken, different distributions of environment frequencies can lead to the same irreversible evolution. However, when we move away from it, the dynamics departs from irreversibility in qualitatively different manners.
△ Less
Submitted 9 August, 2012;
originally announced August 2012.
