-
Textual Toxicity in Social Media: Understanding the Bangla Toxic Language Expressed in Facebook Comment
Authors:
Mohammad Mamun Or Rashid
Abstract:
Social Media is a repository of digital literature including user-generated content. The users of social media are expressing their opinion with diverse mediums such as text, emojis, memes, and also through other visual and textual mediums. A major portion of these media elements could be treated as harmful to others and they are known by many words including Cyberbullying and Toxic Language . The…
▽ More
Social Media is a repository of digital literature including user-generated content. The users of social media are expressing their opinion with diverse mediums such as text, emojis, memes, and also through other visual and textual mediums. A major portion of these media elements could be treated as harmful to others and they are known by many words including Cyberbullying and Toxic Language . The goal of this research paper is to analyze a curated and value-added dataset of toxic language titled ToxLex_bn . It is an exhaustive wordlist that can be used as classifier material to detect toxicity in social media. The toxic language/script used by the Bengali community as cyberbullying, hate speech and moral policing became major trends in social media culture in Bangladesh and West Bengal. The toxicity became so high that the victims has to post as a counter or release explanation video for the haters. Most cases are pointed to women celebrity and their relation, dress, lifestyle are became trolled and toxicity flooded in comments boxes. Not only celebrity bashing but also hates occurred between Hindu Muslims, India-Bangladesh, Two opponents of 1971 and these are very common for virtual conflict in the comment thread. Even many times facebook comment causes sue and legal matters in Bangladesh and thus it requires more study. In this study, a Bangla toxic language dataset has been analyzed which was inputted by the user in Bengali script & language. For this, about 1968 unique bigrams or phrases as wordlists have been analyzed which are derived from 2207590 comments. It is assumed that this analysis will reinforce the detection of Bangla's toxic language used in social media and thus cure this virtual disease.
△ Less
Submitted 9 December, 2023;
originally announced December 2023.
-
BanLemma: A Word Formation Dependent Rule and Dictionary Based Bangla Lemmatizer
Authors:
Sadia Afrin,
Md. Shahad Mahmud Chowdhury,
Md. Ekramul Islam,
Faisal Ahamed Khan,
Labib Imam Chowdhury,
MD. Motahar Mahtab,
Nazifa Nuha Chowdhury,
Massud Forkan,
Neelima Kundu,
Hakim Arif,
Mohammad Mamun Or Rashid,
Mohammad Ruhul Amin,
Nabeel Mohammed
Abstract:
Lemmatization holds significance in both natural language processing (NLP) and linguistics, as it effectively decreases data density and aids in comprehending contextual meaning. However, due to the highly inflected nature and morphological richness, lemmatization in Bangla text poses a complex challenge. In this study, we propose linguistic rules for lemmatization and utilize a dictionary along w…
▽ More
Lemmatization holds significance in both natural language processing (NLP) and linguistics, as it effectively decreases data density and aids in comprehending contextual meaning. However, due to the highly inflected nature and morphological richness, lemmatization in Bangla text poses a complex challenge. In this study, we propose linguistic rules for lemmatization and utilize a dictionary along with the rules to design a lemmatizer specifically for Bangla. Our system aims to lemmatize words based on their parts of speech class within a given sentence. Unlike previous rule-based approaches, we analyzed the suffix marker occurrence according to the morpho-syntactic values and then utilized sequences of suffix markers instead of entire suffixes. To develop our rules, we analyze a large corpus of Bangla text from various domains, sources, and time periods to observe the word formation of inflected words. The lemmatizer achieves an accuracy of 96.36% when tested against a manually annotated test dataset by trained linguists and demonstrates competitive performance on three previously published Bangla lemmatization datasets. We are making the code and datasets publicly available at https://github.com/eblict-gigatech/BanLemma in order to contribute to the further advancement of Bangla NLP.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
DePAint: A Decentralized Safe Multi-Agent Reinforcement Learning Algorithm considering Peak and Average Constraints
Authors:
Raheeb Hassan,
K. M. Shadman Wadith,
Md. Mamun or Rashid,
Md. Mosaddek Khan
Abstract:
The domain of safe multi-agent reinforcement learning (MARL), despite its potential applications in areas ranging from drone delivery and vehicle automation to the development of zero-energy communities, remains relatively unexplored. The primary challenge involves training agents to learn optimal policies that maximize rewards while adhering to stringent safety constraints, all without the oversi…
▽ More
The domain of safe multi-agent reinforcement learning (MARL), despite its potential applications in areas ranging from drone delivery and vehicle automation to the development of zero-energy communities, remains relatively unexplored. The primary challenge involves training agents to learn optimal policies that maximize rewards while adhering to stringent safety constraints, all without the oversight of a central controller. These constraints are critical in a wide array of applications. Moreover, ensuring the privacy of sensitive information in decentralized settings introduces an additional layer of complexity, necessitating innovative solutions that uphold privacy while achieving the system's safety and efficiency goals. In this paper, we address the problem of multi-agent policy optimization in a decentralized setting, where agents communicate with their neighbors to maximize the sum of their cumulative rewards while also satisfying each agent's safety constraints. We consider both peak and average constraints. In this scenario, there is no central controller coordinating the agents and both the rewards and constraints are only known to each agent locally/privately. We formulate the problem as a decentralized constrained multi-agent Markov Decision Problem and propose a momentum-based decentralized policy gradient method, DePAint, to solve it. To the best of our knowledge, this is the first privacy-preserving fully decentralized multi-agent reinforcement learning algorithm that considers both peak and average constraints. We then provide theoretical analysis and empirical evaluation of our algorithm in a number of scenarios and compare its performance to centralized algorithms that consider similar constraints.
△ Less
Submitted 3 April, 2024; v1 submitted 22 October, 2023;
originally announced October 2023.
-
SentiGOLD: A Large Bangla Gold Standard Multi-Domain Sentiment Analysis Dataset and its Evaluation
Authors:
Md. Ekramul Islam,
Labib Chowdhury,
Faisal Ahamed Khan,
Shazzad Hossain,
Sourave Hossain,
Mohammad Mamun Or Rashid,
Nabeel Mohammed,
Mohammad Ruhul Amin
Abstract:
This study introduces SentiGOLD, a Bangla multi-domain sentiment analysis dataset. Comprising 70,000 samples, it was created from diverse sources and annotated by a gender-balanced team of linguists. SentiGOLD adheres to established linguistic conventions agreed upon by the Government of Bangladesh and a Bangla linguistics committee. Unlike English and other languages, Bangla lacks standard sentim…
▽ More
This study introduces SentiGOLD, a Bangla multi-domain sentiment analysis dataset. Comprising 70,000 samples, it was created from diverse sources and annotated by a gender-balanced team of linguists. SentiGOLD adheres to established linguistic conventions agreed upon by the Government of Bangladesh and a Bangla linguistics committee. Unlike English and other languages, Bangla lacks standard sentiment analysis datasets due to the absence of a national linguistics framework. The dataset incorporates data from online video comments, social media posts, blogs, news, and other sources while maintaining domain and class distribution rigorously. It spans 30 domains (e.g., politics, entertainment, sports) and includes 5 sentiment classes (strongly negative, weakly negative, neutral, and strongly positive). The annotation scheme, approved by the national linguistics committee, ensures a robust Inter Annotator Agreement (IAA) with a Fleiss' kappa score of 0.88. Intra- and cross-dataset evaluation protocols are applied to establish a standard classification system. Cross-dataset evaluation on the noisy SentNoB dataset presents a challenging test scenario. Additionally, zero-shot experiments demonstrate the generalizability of SentiGOLD. The top model achieves a macro f1 score of 0.62 (intra-dataset) across 5 classes, setting a benchmark, and 0.61 (cross-dataset from SentNoB) across 3 classes, comparable to the state-of-the-art. Fine-tuned sentiment analysis model can be accessed at https://sentiment.bangla.gov.bd.
△ Less
Submitted 9 June, 2023;
originally announced June 2023.
-
Unicode Normalization and Grapheme Parsing of Indic Languages
Authors:
Nazmuddoha Ansary,
Quazi Adibur Rahman Adib,
Tahsin Reasat,
Asif Shahriyar Sushmit,
Ahmed Imtiaz Humayun,
Sazia Mehnaz,
Kanij Fatema,
Mohammad Mamun Or Rashid,
Farig Sadeque
Abstract:
Writing systems of Indic languages have orthographic syllables, also known as complex graphemes, as unique horizontal units. A prominent feature of these languages is these complex grapheme units that comprise consonants/consonant conjuncts, vowel diacritics, and consonant diacritics, which, together make a unique Language. Unicode-based writing schemes of these languages often disregard this feat…
▽ More
Writing systems of Indic languages have orthographic syllables, also known as complex graphemes, as unique horizontal units. A prominent feature of these languages is these complex grapheme units that comprise consonants/consonant conjuncts, vowel diacritics, and consonant diacritics, which, together make a unique Language. Unicode-based writing schemes of these languages often disregard this feature of these languages and encode words as linear sequences of Unicode characters using an intricate scheme of connector characters and font interpreters. Due to this way of using a few dozen Unicode glyphs to write thousands of different unique glyphs (complex graphemes), there are serious ambiguities that lead to malformed words. In this paper, we are proposing two libraries: i) a normalizer for normalizing inconsistencies caused by a Unicode-based encoding scheme for Indic languages and ii) a grapheme parser for Abugida text. It deconstructs words into visually distinct orthographic syllables or complex graphemes and their constituents. Our proposed normalizer is a more efficient and effective tool than the previously used IndicNLP normalizer. Moreover, our parser and normalizer are also suitable tools for general Abugida text processing as they performed well in our robust word-based and NLP experiments. We report the pipeline for the scripts of 7 languages in this work and develop the framework for the integration of more scripts.
△ Less
Submitted 27 May, 2024; v1 submitted 11 May, 2023;
originally announced June 2023.
-
BenCoref: A Multi-Domain Dataset of Nominal Phrases and Pronominal Reference Annotations
Authors:
Shadman Rohan,
Mojammel Hossain,
Mohammad Mamun Or Rashid,
Nabeel Mohammed
Abstract:
Coreference Resolution is a well studied problem in NLP. While widely studied for English and other resource-rich languages, research on coreference resolution in Bengali largely remains unexplored due to the absence of relevant datasets. Bengali, being a low-resource language, exhibits greater morphological richness compared to English. In this article, we introduce a new dataset, BenCoref, compr…
▽ More
Coreference Resolution is a well studied problem in NLP. While widely studied for English and other resource-rich languages, research on coreference resolution in Bengali largely remains unexplored due to the absence of relevant datasets. Bengali, being a low-resource language, exhibits greater morphological richness compared to English. In this article, we introduce a new dataset, BenCoref, comprising coreference annotations for Bengali texts gathered from four distinct domains. This relatively small dataset contains 5200 mention annotations forming 502 mention clusters within 48,569 tokens. We describe the process of creating this dataset and report performance of multiple models trained using BenCoref. We expect that our work provides some valuable insights on the variations in coreference phenomena across several domains in Bengali and encourages the development of additional resources for Bengali. Furthermore, we found poor crosslingual performance at zero-shot setting from English, highlighting the need for more language-specific resources for this task.
△ Less
Submitted 3 July, 2023; v1 submitted 7 April, 2023;
originally announced April 2023.
-
Implementation of the open source virtualization technologies in cloud computing
Authors:
Mohammad Mamun Or Rashid,
M. Masud Rana,
Jugal Krishna Das
Abstract:
The Virtualization and Cloud Computing is a recent buzzword in the digital world. Cloud computing provide IT as a service to the users on demand basis. This service has greater flexibility, availability, reliability and scalability with utility computing model. This new concept of computing has an immense potential in it to be used in the field of e-governance and in the overall IT development per…
▽ More
The Virtualization and Cloud Computing is a recent buzzword in the digital world. Cloud computing provide IT as a service to the users on demand basis. This service has greater flexibility, availability, reliability and scalability with utility computing model. This new concept of computing has an immense potential in it to be used in the field of e-governance and in the overall IT development perspective in develo** countries like Bangladesh.
△ Less
Submitted 11 May, 2016;
originally announced May 2016.
