-
Toxicity Classification in Ukrainian
Authors:
Daryna Dementieva,
Valeriia Khylenko,
Nikolay Babakov,
Georg Groh
Abstract:
The task of toxicity detection is still a relevant task, especially in the context of safe and fair LMs development. Nevertheless, labeled binary toxicity classification corpora are not available for all languages, which is understandable given the resource-intensive nature of the annotation process. Ukrainian, in particular, is among the languages lacking such resources. To our knowledge, there h…
▽ More
The task of toxicity detection is still a relevant task, especially in the context of safe and fair LMs development. Nevertheless, labeled binary toxicity classification corpora are not available for all languages, which is understandable given the resource-intensive nature of the annotation process. Ukrainian, in particular, is among the languages lacking such resources. To our knowledge, there has been no existing toxicity classification corpus in Ukrainian. In this study, we aim to fill this gap by investigating cross-lingual knowledge transfer techniques and creating labeled corpora by: (i)~translating from an English corpus, (ii)~filtering toxic samples using keywords, and (iii)~annotating with crowdsourcing. We compare LLMs prompting and other cross-lingual transfer approaches with and without fine-tuning offering insights into the most robust and efficient baselines.
△ Less
Submitted 27 April, 2024;
originally announced April 2024.
-
Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora?
Authors:
Miriam Anschütz,
Edoardo Mosca,
Georg Groh
Abstract:
Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionall…
▽ More
Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionally, we conduct a detailed analysis of the correlation between prediction change rates and simplification types/strengths. Our findings reveal alarming inconsistencies across all languages and models. If not promptly addressed, simplified inputs can be easily exploited to craft zero-iteration model-agnostic adversarial attacks with success rates of up to 50%
△ Less
Submitted 10 April, 2024;
originally announced April 2024.
-
Ukrainian Texts Classification: Exploration of Cross-lingual Knowledge Transfer Approaches
Authors:
Daryna Dementieva,
Valeriia Khylenko,
Georg Groh
Abstract:
Despite the extensive amount of labeled datasets in the NLP text classification field, the persistent imbalance in data availability across various languages remains evident. Ukrainian, in particular, stands as a language that still can benefit from the continued refinement of cross-lingual methodologies. Due to our knowledge, there is a tremendous lack of Ukrainian corpora for typical text classi…
▽ More
Despite the extensive amount of labeled datasets in the NLP text classification field, the persistent imbalance in data availability across various languages remains evident. Ukrainian, in particular, stands as a language that still can benefit from the continued refinement of cross-lingual methodologies. Due to our knowledge, there is a tremendous lack of Ukrainian corpora for typical text classification tasks. In this work, we leverage the state-of-the-art advances in NLP, exploring cross-lingual knowledge transfer methods avoiding manual data curation: large multilingual encoders and translation systems, LLMs, and language adapters. We test the approaches on three text classification tasks -- toxicity classification, formality classification, and natural language inference -- providing the "recipe" for the optimal setups.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
This is not correct! Negation-aware Evaluation of Language Generation Systems
Authors:
Miriam Anschütz,
Diego Miguel Lozano,
Georg Groh
Abstract:
Large language models underestimate the impact of negations on how much they change the meaning of a sentence. Therefore, learned evaluation metrics based on these models are insensitive to negations. In this paper, we propose NegBLEURT, a negation-aware version of the BLEURT evaluation metric. For that, we designed a rule-based sentence negation tool and used it to create the CANNOT negation eval…
▽ More
Large language models underestimate the impact of negations on how much they change the meaning of a sentence. Therefore, learned evaluation metrics based on these models are insensitive to negations. In this paper, we propose NegBLEURT, a negation-aware version of the BLEURT evaluation metric. For that, we designed a rule-based sentence negation tool and used it to create the CANNOT negation evaluation dataset. Based on this dataset, we fine-tuned a sentence transformer and an evaluation metric to improve their negation sensitivity. Evaluating these models on existing benchmarks shows that our fine-tuned models outperform existing metrics on the negated sentences by far while preserving their base models' performances on other perturbations.
△ Less
Submitted 26 July, 2023;
originally announced July 2023.
-
Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training
Authors:
Miriam Anschütz,
Joshua Oehms,
Thomas Wimmer,
Bartłomiej Jezierski,
Georg Groh
Abstract:
Automatic text simplification systems help to reduce textual information barriers on the internet. However, for languages other than English, only few parallel data to train these systems exists. We propose a two-step approach to overcome this data scarcity issue. First, we fine-tuned language models on a corpus of German Easy Language, a specific style of German. Then, we used these models as dec…
▽ More
Automatic text simplification systems help to reduce textual information barriers on the internet. However, for languages other than English, only few parallel data to train these systems exists. We propose a two-step approach to overcome this data scarcity issue. First, we fine-tuned language models on a corpus of German Easy Language, a specific style of German. Then, we used these models as decoders in a sequence-to-sequence simplification task. We show that the language models adapt to the style characteristics of Easy Language and output more accessible texts. Moreover, with the style-specific pre-training, we reduced the number of trainable parameters in text simplification models. Hence, less parallel data is sufficient for training. Our results indicate that pre-training on unaligned data can reduce the required parallel data while improving the performance on downstream tasks.
△ Less
Submitted 22 May, 2023;
originally announced May 2023.
-
AdamR at SemEval-2023 Task 10: Solving the Class Imbalance Problem in Sexism Detection with Ensemble Learning
Authors:
Adam Rydelek,
Daryna Dementieva,
Georg Groh
Abstract:
The Explainable Detection of Online Sexism task presents the problem of explainable sexism detection through fine-grained categorisation of sexist cases with three subtasks. Our team experimented with different ways to combat class imbalance throughout the tasks using data augmentation and loss alteration techniques. We tackled the challenge by utilising ensembles of Transformer models trained on…
▽ More
The Explainable Detection of Online Sexism task presents the problem of explainable sexism detection through fine-grained categorisation of sexist cases with three subtasks. Our team experimented with different ways to combat class imbalance throughout the tasks using data augmentation and loss alteration techniques. We tackled the challenge by utilising ensembles of Transformer models trained on different datasets, which are tested to find the balance between performance and interpretability. This solution ranked us in the top 40\% of teams for each of the tracks.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Models
Authors:
Daniel Schroter,
Daryna Dementieva,
Georg Groh
Abstract:
This paper presents the best-performing approach alias "Adam Smith" for the SemEval-2023 Task 4: "Identification of Human Values behind Arguments". The goal of the task was to create systems that automatically identify the values within textual arguments. We train transformer-based models until they reach their loss minimum or f1-score maximum. Ensembling the models by selecting one global decisio…
▽ More
This paper presents the best-performing approach alias "Adam Smith" for the SemEval-2023 Task 4: "Identification of Human Values behind Arguments". The goal of the task was to create systems that automatically identify the values within textual arguments. We train transformer-based models until they reach their loss minimum or f1-score maximum. Ensembling the models by selecting one global decision threshold that maximizes the f1-score leads to the best-performing system in the competition. Ensembling based on stacking with logistic regressions shows the best performance on an additional dataset provided to evaluate the robustness ("Nahj al-Balagha"). Apart from outlining the submitted system, we demonstrate that the use of the large ensemble model is not necessary and that the system size can be significantly reduced.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models
Authors:
Edoardo Mosca,
Daryna Dementieva,
Tohid Ebrahim Ajdari,
Maximilian Kummeth,
Kirill Gringauz,
Yutong Zhou,
Georg Groh
Abstract:
Interpretability and human oversight are fundamental pillars of deploying complex NLP models into real-world applications. However, applying explainability and human-in-the-loop methods requires technical proficiency. Despite existing toolkits for model understanding and analysis, options to integrate human feedback are still limited. We propose IFAN, a framework for real-time explanation-based in…
▽ More
Interpretability and human oversight are fundamental pillars of deploying complex NLP models into real-world applications. However, applying explainability and human-in-the-loop methods requires technical proficiency. Despite existing toolkits for model understanding and analysis, options to integrate human feedback are still limited. We propose IFAN, a framework for real-time explanation-based interaction with NLP models. Through IFAN's interface, users can provide feedback to selected model explanations, which is then integrated through adapter layers to align the model with human rationale. We show the system to be effective in debiasing a hate speech classifier with minimal impact on performance. IFAN also offers a visual admin system and API to manage models (and datasets) as well as control access rights. A demo is live at https://ifan.ml.
△ Less
Submitted 2 October, 2023; v1 submitted 6 March, 2023;
originally announced March 2023.
-
From Judgement's Premises Towards Key Points
Authors:
Oren Sultan,
Rayen Dhahri,
Yauheni Mardan,
Tobias Eder,
Georg Groh
Abstract:
Key Point Analysis(KPA) is a relatively new task in NLP that combines summarization and classification by extracting argumentative key points (KPs) for a topic from a collection of texts and categorizing their closeness to the different arguments. In our work, we focus on the legal domain and develop methods that identify and extract KPs from premises derived from texts of judgments. The first met…
▽ More
Key Point Analysis(KPA) is a relatively new task in NLP that combines summarization and classification by extracting argumentative key points (KPs) for a topic from a collection of texts and categorizing their closeness to the different arguments. In our work, we focus on the legal domain and develop methods that identify and extract KPs from premises derived from texts of judgments. The first method is an adaptation to an existing state-of-the-art method, and the two others are new methods that we developed from scratch. We present our methods and examples of their outputs, as well a comparison between them. The full evaluation of our results is done in the matching task -- match between the generated KPs to arguments (premises).
△ Less
Submitted 23 December, 2022;
originally announced December 2022.
-
Retrieving Users' Opinions on Social Media with Multimodal Aspect-Based Sentiment Analysis
Authors:
Miriam Anschütz,
Tobias Eder,
Georg Groh
Abstract:
People post their opinions and experiences on social media, yielding rich databases of end-users' sentiments. This paper shows to what extent machine learning can analyze and structure these databases. An automated data analysis pipeline is deployed to provide insights into user-generated content for researchers in other domains. First, the domain expert can select an image and a term of interest.…
▽ More
People post their opinions and experiences on social media, yielding rich databases of end-users' sentiments. This paper shows to what extent machine learning can analyze and structure these databases. An automated data analysis pipeline is deployed to provide insights into user-generated content for researchers in other domains. First, the domain expert can select an image and a term of interest. Then, the pipeline uses image retrieval to find all images showing similar content and applies aspect-based sentiment analysis to outline users' opinions about the selected term. As part of an interdisciplinary project between architecture and computer science researchers, an empirical study of Hamburg's Elbphilharmonie was conveyed. Therefore, we selected 300 thousand posts with the hashtag \enquote{\texttt{hamburg}} from the platform Flickr. Image retrieval methods generated a subset of slightly more than 1.5 thousand images displaying the Elbphilharmonie. We found that these posts mainly convey a neutral or positive sentiment towards it. With this pipeline, we suggest a new semantic computing method that offers novel insights into end-users opinions, e.g., for architecture domain experts.
△ Less
Submitted 9 January, 2023; v1 submitted 27 October, 2022;
originally announced October 2022.
-
"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks
Authors:
Edoardo Mosca,
Shreyash Agarwal,
Javier Rando,
Georg Groh
Abstract:
Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work pres…
▽ More
Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work presents a model-agnostic detector of adversarial text examples. The approach identifies patterns in the logits of the target classifier when perturbing the input text. The proposed detector improves the current state-of-the-art performance in recognizing adversarial inputs and exhibits strong generalization capabilities across different NLP models, datasets, and word-level attacks.
△ Less
Submitted 29 June, 2023; v1 submitted 10 April, 2022;
originally announced April 2022.
-
How to Build Robust FAQ Chatbot with Controllable Question Generator?
Authors:
Yan Pan,
Mingyang Ma,
Bernhard Pflugfelder,
Georg Groh
Abstract:
Many unanswerable adversarial questions fool the question-answer (QA) system with some plausible answers. Building a robust, frequently asked questions (FAQ) chatbot needs a large amount of diverse adversarial examples. Recent question generation methods are ineffective at generating many high-quality and diverse adversarial question-answer pairs from unstructured text. We propose the diversity co…
▽ More
Many unanswerable adversarial questions fool the question-answer (QA) system with some plausible answers. Building a robust, frequently asked questions (FAQ) chatbot needs a large amount of diverse adversarial examples. Recent question generation methods are ineffective at generating many high-quality and diverse adversarial question-answer pairs from unstructured text. We propose the diversity controllable semantically valid adversarial attacker (DCSA), a high-quality, diverse, controllable method to generate standard and adversarial samples with a semantic graph. The fluent and semantically generated QA pairs fool our passage retrieval model successfully. After that, we conduct a study on the robustness and generalization of the QA model with generated QA pairs among different domains. We find that the generated data set improves the generalizability of the QA model to the new target domain and the robustness of the QA model to detect unanswerable adversarial questions.
△ Less
Submitted 18 November, 2021;
originally announced December 2021.
-
End-to-End Annotator Bias Approximation on Crowdsourced Single-Label Sentiment Analysis
Authors:
Gerhard Johann Hagerer,
David Szabo,
Andreas Koch,
Maria Luisa Ripoll Dominguez,
Christian Widmer,
Maximilian Wich,
Hannah Danner,
Georg Groh
Abstract:
Sentiment analysis is often a crowdsourcing task prone to subjective labels given by many annotators. It is not yet fully understood how the annotation bias of each annotator can be modeled correctly with state-of-the-art methods. However, resolving annotator bias precisely and reliably is the key to understand annotators' labeling behavior and to successfully resolve corresponding individual misc…
▽ More
Sentiment analysis is often a crowdsourcing task prone to subjective labels given by many annotators. It is not yet fully understood how the annotation bias of each annotator can be modeled correctly with state-of-the-art methods. However, resolving annotator bias precisely and reliably is the key to understand annotators' labeling behavior and to successfully resolve corresponding individual misconceptions and wrongdoings regarding the annotation task. Our contribution is an explanation and improvement for precise neural end-to-end bias modeling and ground truth estimation, which reduces an undesired mismatch in that regard of the existing state-of-the-art. Classification experiments show that it has potential to improve accuracy in cases where each sample is annotated only by one single annotator. We provide the whole source code publicly and release an own domain-specific sentiment dataset containing 10,000 sentences discussing organic food products. These are crawled from social media and are singly labeled by 10 non-expert annotators.
△ Less
Submitted 24 July, 2023; v1 submitted 3 November, 2021;
originally announced November 2021.
-
A Case Study and Qualitative Analysis of Simple Cross-Lingual Opinion Mining
Authors:
Gerhard Johann Hagerer,
Wing Sheung Leung,
Qiaoxi Liu,
Hannah Danner,
Georg Groh
Abstract:
User-generated content from social media is produced in many languages, making it technically challenging to compare the discussed themes from one domain across different cultures and regions. It is relevant for domains in a globalized world, such as market research, where people from two nations and markets might have different requirements for a product. We propose a simple, modern, and effectiv…
▽ More
User-generated content from social media is produced in many languages, making it technically challenging to compare the discussed themes from one domain across different cultures and regions. It is relevant for domains in a globalized world, such as market research, where people from two nations and markets might have different requirements for a product. We propose a simple, modern, and effective method for building a single topic model with sentiment analysis capable of covering multiple languages simultanteously, based on a pre-trained state-of-the-art deep neural network for natural language understanding. To demonstrate its feasibility, we apply the model to newspaper articles and user comments of a specific domain, i.e., organic food products and related consumption behavior. The themes match across languages. Additionally, we obtain an high proportion of stable and domain-relevant topics, a meaningful relation between topics and their respective textual contents, and an interpretable representation for social media documents. Marketing can potentially benefit from our method, since it provides an easy-to-use means of addressing specific customer interests from different market regions around the globe. For reproducibility, we provide the code, data, and results of our study.
△ Less
Submitted 24 July, 2023; v1 submitted 3 November, 2021;
originally announced November 2021.
-
An Analysis of Programming Course Evaluations Before and After the Introduction of an Autograder
Authors:
Gerhard Johann Hagerer,
Laura Lahesoo,
Miriam Anschütz,
Stephan Krusche,
Georg Groh
Abstract:
Commonly, introductory programming courses in higher education institutions have hundreds of participating students eager to learn to program. The manual effort for reviewing the submitted source code and for providing feedback can no longer be managed. Manually reviewing the submitted homework can be subjective and unfair, particularly if many tutors are responsible for grading. Different autogra…
▽ More
Commonly, introductory programming courses in higher education institutions have hundreds of participating students eager to learn to program. The manual effort for reviewing the submitted source code and for providing feedback can no longer be managed. Manually reviewing the submitted homework can be subjective and unfair, particularly if many tutors are responsible for grading. Different autograders can help in this situation; however, there is a lack of knowledge about how autograders can impact students' overall perception of programming classes and teaching. This is relevant for course organizers and institutions to keep their programming courses attractive while co** with increasing students.
This paper studies the answers to the standardized university evaluation questionnaires of multiple large-scale foundational computer science courses which recently introduced autograding. The differences before and after this intervention are analyzed. By incorporating additional observations, we hypothesize how the autograder might have contributed to the significant changes in the data, such as, improved interactions between tutors and students, improved overall course quality, improved learning success, increased time spent, and reduced difficulty. This qualitative study aims to provide hypotheses for future research to define and conduct quantitative surveys and data analysis. The autograder technology can be validated as a teaching method to improve student satisfaction with programming courses.
△ Less
Submitted 24 July, 2023; v1 submitted 28 October, 2021;
originally announced October 2021.
-
SocialVisTUM: An Interactive Visualization Toolkit for Correlated Neural Topic Models on Social Media Opinion Mining
Authors:
Gerhard Johann Hagerer,
Martin Kirchhoff,
Hannah Danner,
Robert Pesch,
Mainak Ghosh,
Archishman Roy,
Jiaxi Zhao,
Georg Groh
Abstract:
Recent research in opinion mining proposed word embedding-based topic modeling methods that provide superior coherence compared to traditional topic modeling. In this paper, we demonstrate how these methods can be used to display correlated topic models on social media texts using SocialVisTUM, our proposed interactive visualization toolkit. It displays a graph with topics as nodes and their corre…
▽ More
Recent research in opinion mining proposed word embedding-based topic modeling methods that provide superior coherence compared to traditional topic modeling. In this paper, we demonstrate how these methods can be used to display correlated topic models on social media texts using SocialVisTUM, our proposed interactive visualization toolkit. It displays a graph with topics as nodes and their correlations as edges. Further details are displayed interactively to support the exploration of large text collections, e.g., representative words and sentences of topics, topic and sentiment distributions, hierarchical topic clustering, and customizable, predefined topic labels. The toolkit optimizes automatically on custom data for optimal coherence. We show a working instance of the toolkit on data crawled from English social media discussions about organic food consumption. The visualization confirms findings of a qualitative consumer research study. SocialVisTUM and its training procedures are accessible online.
△ Less
Submitted 24 July, 2023; v1 submitted 20 October, 2021;
originally announced October 2021.
-
Introducing an Abusive Language Classification Framework for Telegram to Investigate the German Hater Community
Authors:
Maximilian Wich,
Adrian Gorniak,
Tobias Eder,
Daniel Bartmann,
Burak Enes Çakici,
Georg Groh
Abstract:
Since traditional social media platforms continue to ban actors spreading hate speech or other forms of abusive languages (a process known as deplatforming), these actors migrate to alternative platforms that do not moderate users content. One popular platform relevant for the German hater community is Telegram for which limited research efforts have been made so far. This study aims to develop a…
▽ More
Since traditional social media platforms continue to ban actors spreading hate speech or other forms of abusive languages (a process known as deplatforming), these actors migrate to alternative platforms that do not moderate users content. One popular platform relevant for the German hater community is Telegram for which limited research efforts have been made so far. This study aims to develop a broad framework comprising (i) an abusive language classification model for German Telegram messages and (ii) a classification model for the hatefulness of Telegram channels. For the first part, we use existing abusive language datasets containing posts from other platforms to develop our classification models. For the channel classification model, we develop a method that combines channel-specific content information collected from a topic model with a social graph to predict the hatefulness of channels. Furthermore, we complement these two approaches for hate speech detection with insightful results on the evolution of the hater community on Telegram in Germany. We also propose methods for conducting scalable network analyses for social media platforms to the hate speech research community. As an additional output of this study, we provide an annotated abusive language dataset containing 1,149 annotated Telegram messages.
△ Less
Submitted 24 November, 2021; v1 submitted 15 September, 2021;
originally announced September 2021.
-
Classification of Consumer Belief Statements From Social Media
Authors:
Gerhard Johann Hagerer,
Wenbin Le,
Hannah Danner,
Georg Groh
Abstract:
Social media offer plenty of information to perform market research in order to meet the requirements of customers. One way how this research is conducted is that a domain expert gathers and categorizes user-generated content into a complex and fine-grained class structure. In many of such cases, little data meets complex annotations. It is not yet fully understood how this can be leveraged succes…
▽ More
Social media offer plenty of information to perform market research in order to meet the requirements of customers. One way how this research is conducted is that a domain expert gathers and categorizes user-generated content into a complex and fine-grained class structure. In many of such cases, little data meets complex annotations. It is not yet fully understood how this can be leveraged successfully for classification. We examine the classification accuracy of expert labels when used with a) many fine-grained classes and b) few abstract classes. For scenario b) we compare abstract class labels given by the domain expert as baseline and by automatic hierarchical clustering. We compare this to another baseline where the entire class structure is given by a completely unsupervised clustering approach. By doing so, this work can serve as an example of how complex expert annotations are potentially beneficial and can be utilized in the most optimal way for opinion mining in highly specific domains. By exploring across a range of techniques and experiments, we find that automated class abstraction approaches in particular the unsupervised approach performs remarkably well against domain expert baseline on text classification tasks. This has the potential to inspire opinion mining applications in order to support market researchers in practice and to inspire fine-grained automated content analysis on a large scale.
△ Less
Submitted 24 July, 2023; v1 submitted 29 June, 2021;
originally announced June 2021.
-
GraphTMT: Unsupervised Graph-based Topic Modeling from Video Transcripts
Authors:
Lukas Stappen,
Jason Thies,
Gerhard Hagerer,
Björn W. Schuller,
Georg Groh
Abstract:
To unfold the tremendous amount of multimedia data uploaded daily to social media platforms, effective topic modeling techniques are needed. Existing work tends to apply topic models on written text datasets. In this paper, we propose a topic extractor on video transcripts. Exploiting neural word embeddings through graph-based clustering, we aim to improve usability and semantic coherence. Unlike…
▽ More
To unfold the tremendous amount of multimedia data uploaded daily to social media platforms, effective topic modeling techniques are needed. Existing work tends to apply topic models on written text datasets. In this paper, we propose a topic extractor on video transcripts. Exploiting neural word embeddings through graph-based clustering, we aim to improve usability and semantic coherence. Unlike most topic models, this approach works without knowing the true number of topics, which is important when no such assumption can or should be made. Experimental results on the real-life multimodal dataset MuSe-CaR demonstrates that our approach GraphTMT extracts coherent and meaningful topics and outperforms baseline methods. Furthermore, we successfully demonstrate the applicability of our approach on the popular Citysearch corpus.
△ Less
Submitted 28 October, 2021; v1 submitted 4 May, 2021;
originally announced May 2021.
-
Contributive Social Capital Extraction From Different Types of Online Data Sources
Authors:
Sebastian Schams,
Georg Groh
Abstract:
It is a recurring problem of online communication that the properties of unknown people are hard to assess. This may lead to various issues such as the spread of `fake news' from untrustworthy sources. In sociology the sum of (social) resources available to a person through their social network is often described as social capital. In this article, we look at social capital from a different angle.…
▽ More
It is a recurring problem of online communication that the properties of unknown people are hard to assess. This may lead to various issues such as the spread of `fake news' from untrustworthy sources. In sociology the sum of (social) resources available to a person through their social network is often described as social capital. In this article, we look at social capital from a different angle. Instead of evaluating the advantage that people have because of their membership in a certain group, we investigate various ways to infer the social capital a person adds or may add to the network, their contributive social capital (CSC). As there is no consensus in the literature on what the social capital of a person exactly consists of, we look at various related properties: expertise, reputation, trustworthiness, and influence. The analysis of these features is investigated for five different sources of online data: microblogging (e.g., Twitter), social networking platforms (e.g., Facebook), direct communication (e.g., email), scientometrics, and threaded discussion boards (e.g., Reddit). In each field we discuss recent publications and put a focus on the data sources used, the algorithms implemented, and the performance evaluation. The findings are compared and set in context to contributive social capital extraction. The analysis algorithms are based on individual features (e.g., followers on Twitter), ratios thereof, or a person's centrality measures (e.g., PageRank). The machine learning approaches, such as straightforward classifiers (e.g., support vector machines) use ground truths that are connected to social capital. The discussion of these methods is intended to facilitate research on the topic by identifying relevant data sources and the best suited algorithms, and by providing tested methods for the evaluation of findings.
△ Less
Submitted 20 February, 2019;
originally announced February 2019.
-
Sequence Labeling: A Practical Approach
Authors:
Adnan Akhundov,
Dietrich Trautmann,
Georg Groh
Abstract:
We take a practical approach to solving sequence labeling problem assuming unavailability of domain expertise and scarcity of informational and computational resources. To this end, we utilize a universal end-to-end Bi-LSTM-based neural sequence labeling model applicable to a wide range of NLP tasks and languages. The model combines morphological, semantic, and structural cues extracted from data…
▽ More
We take a practical approach to solving sequence labeling problem assuming unavailability of domain expertise and scarcity of informational and computational resources. To this end, we utilize a universal end-to-end Bi-LSTM-based neural sequence labeling model applicable to a wide range of NLP tasks and languages. The model combines morphological, semantic, and structural cues extracted from data to arrive at informed predictions. The model's performance is evaluated on eight benchmark datasets (covering three tasks: POS-tagging, NER, and Chunking, and four languages: English, German, Dutch, and Spanish). We observe state-of-the-art results on four of them: CoNLL-2012 (English NER), CoNLL-2002 (Dutch NER), GermEval 2014 (German NER), Tiger Corpus (German POS-tagging), and competitive performance on the rest.
△ Less
Submitted 12 August, 2018;
originally announced August 2018.
-
Estimating the Dissemination of Social and Mobile Search in Categories of Information Needs Using Websites as Proxies
Authors:
Christoph Fuchs,
Akash Nayyar,
Ruth Nussbaumer,
Georg Groh
Abstract:
With the increasing popularity of social means to satisfy information needs using Social Media (e.g., Social Media Question Asking, SMQA) or Social Information Retrieval approaches, this paper tries to identify types of information needs which are inherently social and therefore better suited for those techniques. We describe an experiment where prominent websites from various content categories a…
▽ More
With the increasing popularity of social means to satisfy information needs using Social Media (e.g., Social Media Question Asking, SMQA) or Social Information Retrieval approaches, this paper tries to identify types of information needs which are inherently social and therefore better suited for those techniques. We describe an experiment where prominent websites from various content categories are used to represent their respective content area and allow to correlate attributes of the content areas. The underlying assumption is that successful websites for focused content areas perfectly align with the information seekers' requirements when satisfying information needs in the respective content areas. Based on a manually collected dataset of URLs from websites covering a broad range of topics taken from Alexa (http://www.alexa.com} (retrieved 2015-11-04)) (a company that publishes statistics about web traffic), a crowdsourcing approach is employed to rate the information needs that could get solved by the respective URLs according to several dimensions (incl. sociality and mobility) to investigate possible correlations with other attributes. Our results suggest that information needs which do not require a certain formal expertise play an important role in social information retrieval and that some content areas are better suited for social information retrieval (e.g., Factual Knowledge & News, Games, Lifestyle) than others (e.g., Health & Lifestyle).
△ Less
Submitted 7 July, 2016;
originally announced July 2016.
-
Mobile Homophily and Social Location Prediction
Authors:
Halgurt Bapierre,
Chakajkla Jesdabodi,
Georg Groh
Abstract:
The mobility behavior of human beings is predictable to a varying degree e.g. depending on the traits of their personality such as the trait extraversion - introversion: the mobility of introvert users may be more dominated by routines and habitual movement patterns, resulting in a more predictable mobility behavior on the basis of their own location history while, in contrast, extrovert users get…
▽ More
The mobility behavior of human beings is predictable to a varying degree e.g. depending on the traits of their personality such as the trait extraversion - introversion: the mobility of introvert users may be more dominated by routines and habitual movement patterns, resulting in a more predictable mobility behavior on the basis of their own location history while, in contrast, extrovert users get about a lot and are explorative by nature, which may hamper the prediction of their mobility. However, socially more active and extrovert users meet more people and share information, experiences, believes, thoughts etc. with others. which in turn leads to a high interdependency between their mobility and social lives. Using a large LBSN dataset, his paper investigates the interdependency between human mobility and social proximity, the influence of social networks on enhancing location prediction of an individual and the transmission of social trends/influences within social networks.
△ Less
Submitted 25 June, 2015;
originally announced June 2015.
-
Reaching Consensus Among Mobile Agents: A Distributed Protocol for the Detection of Social Situations
Authors:
Daniel Raumer,
Christoph Fuchs,
Georg Groh
Abstract:
Physical social encounters are governed by a set of socio-psychological behavioral rules with a high degree of uniform validity. Past research has shown how these rules or the resulting properties of the encounters (e.g. the geometry of interaction) can be used for algorithmic detection of social interaction. In this paper, we present a distributed protocol to gain a common understanding of the ex…
▽ More
Physical social encounters are governed by a set of socio-psychological behavioral rules with a high degree of uniform validity. Past research has shown how these rules or the resulting properties of the encounters (e.g. the geometry of interaction) can be used for algorithmic detection of social interaction. In this paper, we present a distributed protocol to gain a common understanding of the existing social situations among agents.
Our approach allows a group of agents to combine their subjective assessment of an ongoing social situation. Based on perceived social cues obtained from raw data signals, they reach a consensus about the existence, parameters, and participants of a social situation. We evaluate our protocol using two real-world datasets with social interaction information and additional synthetic data generated by our social-aware mobility model.
△ Less
Submitted 29 September, 2014;
originally announced September 2014.
-
Designing Sound Collaboratively - Perceptually Motivated Audio Synthesis
Authors:
Niklas Klügel,
Timo Becker,
Georg Groh
Abstract:
In this contribution, we will discuss a prototype that allows a group of users to design sound collaboratively in real time using a multi-touch tabletop. We make use of a machine learning method to generate a map** from perceptual audio features to synthesis parameters. This map** is then used for visualization and interaction. Finally, we discuss the results of a comparative evaluation study.
In this contribution, we will discuss a prototype that allows a group of users to design sound collaboratively in real time using a multi-touch tabletop. We make use of a machine learning method to generate a map** from perceptual audio features to synthesis parameters. This map** is then used for visualization and interaction. Finally, we discuss the results of a comparative evaluation study.
△ Less
Submitted 23 June, 2014;
originally announced June 2014.
-
An evaluation of keyword extraction from online communication for the characterisation of social relations
Authors:
Jan Hauffa,
Tobias Lichtenberg,
Georg Groh
Abstract:
The set of interpersonal relationships on a social network service or a similar online community is usually highly heterogenous. The concept of tie strength captures only one aspect of this heterogeneity. Since the unstructured text content of online communication artefacts is a salient source of information about a social relationship, we investigate the utility of keywords extracted from the mes…
▽ More
The set of interpersonal relationships on a social network service or a similar online community is usually highly heterogenous. The concept of tie strength captures only one aspect of this heterogeneity. Since the unstructured text content of online communication artefacts is a salient source of information about a social relationship, we investigate the utility of keywords extracted from the message body as a representation of the relationship's characteristics as reflected by the conversation topics. Keyword extraction is performed using standard natural language processing methods. Communication data and human assessments of the extracted keywords are obtained from Facebook users via a custom application. The overall positive quality assessment provides evidence that the keywords indeed convey relevant information about the relationship.
△ Less
Submitted 11 February, 2014;
originally announced February 2014.
-
Spatio-Temporal Small Worlds for Decentralized Information Retrieval in Social Networking
Authors:
Georg Groh,
Florian Straub,
Benjamin Koster
Abstract:
We discuss foundations and options for alternative, agent-based information retrieval (IR) approaches in Social Networking, especially Decentralized and Mobile Social Networking scenarios. In addition to usual semantic contexts, these approaches make use of long-term social and spatio-temporal contexts in order to satisfy conscious as well as unconscious information needs according to Human IR heu…
▽ More
We discuss foundations and options for alternative, agent-based information retrieval (IR) approaches in Social Networking, especially Decentralized and Mobile Social Networking scenarios. In addition to usual semantic contexts, these approaches make use of long-term social and spatio-temporal contexts in order to satisfy conscious as well as unconscious information needs according to Human IR heuristics. Using a large Twitter dataset, we investigate these approaches and especially investigate the question in how far spatio-temporal contexts can act as a conceptual bracket implicating social and semantic cohesion, giving rise to the concept of Spatio-Temporal Small Worlds.
△ Less
Submitted 13 September, 2012;
originally announced September 2012.
-
Interest-Based vs. Social Person-Recommenders in Social Networking Platforms
Authors:
Georg Groh,
Michele Brocco,
Andreas Kleemann
Abstract:
Social network based approaches to person recommendations are compared to interest based approaches with the help of an empirical study on a large German social networking platform. We assess and compare the performance of different basic variants of the two approaches by precision / recall based performance with respect to reproducing known friendship relations and by an empirical questionnaire b…
▽ More
Social network based approaches to person recommendations are compared to interest based approaches with the help of an empirical study on a large German social networking platform. We assess and compare the performance of different basic variants of the two approaches by precision / recall based performance with respect to reproducing known friendship relations and by an empirical questionnaire based study. In accordance to expectation, the results show that interest based person recommenders are able to produce more novel recommendations while performing less well with respect to friendship reproduction. With respect to the user's assessment of recommendation quality all approaches perform comparably well, while combined social-interest-based variants are slightly ahead in performance. The overall results qualify those combined approaches as a good compromise.
△ Less
Submitted 28 July, 2011;
originally announced July 2011.
-
Space and Time as a Primary Classification Criterion for Information Retrieval in Distributed Social Networking
Authors:
Georg Groh,
Florian Straub,
Andreas Donaubauer,
Benjamin Koster
Abstract:
We discuss in a compact way how the implicit relations between spatiotemporal relatedness of information items, spatiotemporal relatedness of users, social relatedness of users and semantic relatedness of information items may be exploited for an information retrieval architecture that operates along the lines of human ways of searching. The decentralized and agent oriented architecture mirrors em…
▽ More
We discuss in a compact way how the implicit relations between spatiotemporal relatedness of information items, spatiotemporal relatedness of users, social relatedness of users and semantic relatedness of information items may be exploited for an information retrieval architecture that operates along the lines of human ways of searching. The decentralized and agent oriented architecture mirrors emerging trends such as upcoming mobile and decentralized social networking as a new paradigm in social computing and is targetted to satisfy broader and more subtly interlinked information demands beyond immediate information needs which can be readily satisfied with current IR services. We briefly discuss why using spatio-temporal references as primary information criterion implicitly conserves other relations and is thus suitable for such an architecture. We finally shortly point to results from a large evaluation study using Wikipedia articles.
△ Less
Submitted 12 April, 2011;
originally announced April 2011.
