-
Interpreting Answers to Yes-No Questions in User-Generated Content
Authors:
Shivam Mathur,
Keun Hee Park,
Dhivya Chinnappa,
Saketh Kotamraju,
Eduardo Blanco
Abstract:
Interpreting answers to yes-no questions in social media is difficult. Yes and no keywords are uncommon, and the few answers that include them are rarely to be interpreted what the keywords suggest. In this paper, we present a new corpus of 4,442 yes-no question-answer pairs from Twitter. We discuss linguistic characteristics of answers whose interpretation is yes or no, as well as answers whose i…
▽ More
Interpreting answers to yes-no questions in social media is difficult. Yes and no keywords are uncommon, and the few answers that include them are rarely to be interpreted what the keywords suggest. In this paper, we present a new corpus of 4,442 yes-no question-answer pairs from Twitter. We discuss linguistic characteristics of answers whose interpretation is yes or no, as well as answers whose interpretation is unknown. We show that large language models are far from solving this problem, even after fine-tuning and blending other corpora for the same problem but outside social media.
△ Less
Submitted 23 October, 2023;
originally announced October 2023.
-
An Analysis of Negation in Natural Language Understanding Corpora
Authors:
Md Mosharaf Hossain,
Dhivya Chinnappa,
Eduardo Blanco
Abstract:
This paper analyzes negation in eight popular corpora spanning six natural language understanding tasks. We show that these corpora have few negations compared to general-purpose English, and that the few negations in them are often unimportant. Indeed, one can often ignore negations and still make the right predictions. Additionally, experimental results show that state-of-the-art transformers tr…
▽ More
This paper analyzes negation in eight popular corpora spanning six natural language understanding tasks. We show that these corpora have few negations compared to general-purpose English, and that the few negations in them are often unimportant. Indeed, one can often ignore negations and still make the right predictions. Additionally, experimental results show that state-of-the-art transformers trained with these corpora obtain substantially worse results with instances that contain negation, especially if the negations are important. We conclude that new corpora accounting for negation are needed to solve natural language understanding tasks when negation is present.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
Findings of the Sentiment Analysis of Dravidian Languages in Code-Mixed Text
Authors:
Bharathi Raja Chakravarthi,
Ruba Priyadharshini,
Sajeetha Thavareesan,
Dhivya Chinnappa,
Durairaj Thenmozhi,
Elizabeth Sherly,
John P. McCrae,
Adeep Hande,
Rahul Ponnusamy,
Shubhanker Banerjee,
Charangan Vasantharajan
Abstract:
We present the results of the Dravidian-CodeMix shared task held at FIRE 2021, a track on sentiment analysis for Dravidian Languages in Code-Mixed Text. We describe the task, its organization, and the submitted systems. This shared task is the continuation of last year's Dravidian-CodeMix shared task held at FIRE 2020. This year's tasks included code-mixing at the intra-token and inter-token level…
▽ More
We present the results of the Dravidian-CodeMix shared task held at FIRE 2021, a track on sentiment analysis for Dravidian Languages in Code-Mixed Text. We describe the task, its organization, and the submitted systems. This shared task is the continuation of last year's Dravidian-CodeMix shared task held at FIRE 2020. This year's tasks included code-mixing at the intra-token and inter-token levels. Additionally, apart from Tamil and Malayalam, Kannada was also introduced. We received 22 systems for Tamil-English, 15 systems for Malayalam-English, and 15 for Kannada-English. The top system for Tamil-English, Malayalam-English and Kannada-English scored weighted average F1-score of 0.711, 0.804, and 0.630, respectively. In summary, the quality and quantity of the submission show that there is great interest in Dravidian languages in code-mixed setting and state of the art in this domain still needs more improvement.
△ Less
Submitted 18 November, 2021;
originally announced November 2021.
-
Develo** Successful Shared Tasks on Offensive Language Identification for Dravidian Languages
Authors:
Bharathi Raja Chakravarthi,
Dhivya Chinnappa,
Ruba Priyadharshini,
Anand Kumar Madasamy,
Sangeetha Sivanesan,
Subalalitha Chinnaudayar Navaneethakrishnan,
Sajeetha Thavareesan,
Dhanalakshmi Vadivel,
Rahul Ponnusamy,
Prasanna Kumar Kumaresan
Abstract:
With the fast growth of mobile computing and Web technologies, offensive language has become more prevalent on social networking platforms. Since offensive language identification in local languages is essential to moderate the social media content, in this paper we work with three Dravidian languages, namely Malayalam, Tamil, and Kannada, that are under-resourced. We present an evaluation task at…
▽ More
With the fast growth of mobile computing and Web technologies, offensive language has become more prevalent on social networking platforms. Since offensive language identification in local languages is essential to moderate the social media content, in this paper we work with three Dravidian languages, namely Malayalam, Tamil, and Kannada, that are under-resourced. We present an evaluation task at FIRE 2020- HASOC-DravidianCodeMix and DravidianLangTech at EACL 2021, designed to provide a framework for comparing different approaches to this problem. This paper describes the data creation, defines the task, lists the participating systems, and discusses various methods.
△ Less
Submitted 5 November, 2021;
originally announced November 2021.
