Showing 1–7 of 7 results for author: Kauermann, G

Human-in-the-loop: Towards Label Embeddings for Measuring Classification Difficulty

Authors: Katharina Hechinger, Christoph Koller, Xiao Xiang Zhu, Göran Kauermann

Abstract: Uncertainty in machine learning models is a timely and vast field of research. In supervised learning, uncertainty can already occur in the first stage of the training process, the annotation phase. This scenario is particularly evident when some instances cannot be definitively classified. In other words, there is inevitable ambiguity in the annotation step and hence, not necessarily a "ground tr… ▽ More Uncertainty in machine learning models is a timely and vast field of research. In supervised learning, uncertainty can already occur in the first stage of the training process, the annotation phase. This scenario is particularly evident when some instances cannot be definitively classified. In other words, there is inevitable ambiguity in the annotation step and hence, not necessarily a "ground truth" associated with each instance. The main idea of this work is to drop the assumption of a ground truth label and instead embed the annotations into a multidimensional space. This embedding is derived from the empirical distribution of annotations in a Bayesian setup, modeled via a Dirichlet-Multinomial framework. We estimate the model parameters and posteriors using a stochastic Expectation Maximization algorithm with Markov Chain Monte Carlo steps. The methods developed in this paper readily extend to various situations where multiple annotators independently label instances. To showcase the generality of the proposed approach, we apply our approach to three benchmark datasets for image classification and Natural Language Inference. Besides the embeddings, we can investigate the resulting correlation matrices, which reflect the semantic similarities of the original classes very well for all three exemplary datasets. △ Less

Submitted 27 May, 2024; v1 submitted 15 November, 2023; originally announced November 2023.

Sources of Uncertainty in Machine Learning -- A Statisticians' View

Authors: Cornelia Gruber, Patrick Oliver Schenk, Malte Schierholz, Frauke Kreuter, Göran Kauermann

Abstract: Machine Learning and Deep Learning have achieved an impressive standard today, enabling us to answer questions that were inconceivable a few years ago. Besides these successes, it becomes clear, that beyond pure prediction, which is the primary strength of most supervised machine learning algorithms, the quantification of uncertainty is relevant and necessary as well. While first concepts and idea… ▽ More Machine Learning and Deep Learning have achieved an impressive standard today, enabling us to answer questions that were inconceivable a few years ago. Besides these successes, it becomes clear, that beyond pure prediction, which is the primary strength of most supervised machine learning algorithms, the quantification of uncertainty is relevant and necessary as well. While first concepts and ideas in this direction have emerged in recent years, this paper adopts a conceptual perspective and examines possible sources of uncertainty. By adopting the viewpoint of a statistician, we discuss the concepts of aleatoric and epistemic uncertainty, which are more commonly associated with machine learning. The paper aims to formalize the two types of uncertainty and demonstrates that sources of uncertainty are miscellaneous and can not always be decomposed into aleatoric and epistemic. Drawing parallels between statistical concepts and uncertainty in machine learning, we also demonstrate the role of data and their influence on uncertainty. △ Less

Submitted 26 May, 2023; originally announced May 2023.

The Politics of Language Choice: How the Russian-Ukrainian War Influences Ukrainians' Language Use on Twitter

Authors: Daniel Racek, Brittany I. Davidson, Paul W. Thurner, Xiao Xiang Zhu, Göran Kauermann

Abstract: The use of language is innately political and often a vehicle of cultural identity as well as the basis for nation building. Here, we examine language choice and tweeting activity of Ukrainian citizens based on more than 4 million geo-tagged tweets from over 62,000 users before and during the Russian-Ukrainian War, from January 2020 to October 2022. Using statistical models, we disentangle sample… ▽ More The use of language is innately political and often a vehicle of cultural identity as well as the basis for nation building. Here, we examine language choice and tweeting activity of Ukrainian citizens based on more than 4 million geo-tagged tweets from over 62,000 users before and during the Russian-Ukrainian War, from January 2020 to October 2022. Using statistical models, we disentangle sample effects, arising from the in- and outflux of users on Twitter, from behavioural effects, arising from behavioural changes of the users. We observe a steady shift from the Russian language towards the Ukrainian language already before the war, which drastically speeds up with its outbreak. We attribute these shifts in large part to users' behavioural changes. Notably, we find that more than half of the Russian-tweeting users shift towards Ukrainian as a result of the war. △ Less

Submitted 6 June, 2023; v1 submitted 4 May, 2023; originally announced May 2023.

COVID-19 and social media: Beyond polarization

Authors: Giacomo De Nicola, Victor H. Tuekam Mambou, Göran Kauermann

Abstract: The COVID-19 pandemic brought upon a massive wave of disinformation, exacerbating polarization in the increasingly divided landscape of online discourse. In this context, popular social media users play a major role, as they have the ability to broadcast messages to large audiences and influence public opinion. In this paper, we make use of openly available data to study the behavior of popular us… ▽ More The COVID-19 pandemic brought upon a massive wave of disinformation, exacerbating polarization in the increasingly divided landscape of online discourse. In this context, popular social media users play a major role, as they have the ability to broadcast messages to large audiences and influence public opinion. In this paper, we make use of openly available data to study the behavior of popular users discussing the pandemic on Twitter. We tackle the issue from a network perspective, considering users as nodes and following relationships as directed edges. The resulting network structure is modeled by embedding the actors in a latent social space, where users closer to one another have a higher probability of following each other. The results suggest the existence of two distinct communities, which can be interpreted as "generally pro" and "generally against" vaccine mandates, corroborating existing evidence on the pervasiveness of echo chambers on the platform. By focusing on a number of notable users, such as politicians, activists, and news outlets, we further show that the two groups are not entirely homogeneous, and that not just the two poles are represented. To the contrary, the latent space captures an entire spectrum of beliefs between the two extremes, demonstrating that polarization, while present, is not the only driver of the network, and that more moderate, "central" users are key players in the discussion. △ Less

Submitted 24 July, 2023; v1 submitted 27 July, 2022; originally announced July 2022.

Going Beyond One-Hot Encoding in Classification: Can Human Uncertainty Improve Model Performance?

Authors: Christoph Koller, Göran Kauermann, Xiao Xiang Zhu

Abstract: Technological and computational advances continuously drive forward the broad field of deep learning. In recent years, the derivation of quantities describing theuncertainty in the prediction - which naturally accompanies the modeling process - has sparked general interest in the deep learning community. Often neglected in the machine learning setting is the human uncertainty that influences numer… ▽ More Technological and computational advances continuously drive forward the broad field of deep learning. In recent years, the derivation of quantities describing theuncertainty in the prediction - which naturally accompanies the modeling process - has sparked general interest in the deep learning community. Often neglected in the machine learning setting is the human uncertainty that influences numerous labeling processes. As the core of this work, label uncertainty is explicitly embedded into the training process via distributional labels. We demonstrate the effectiveness of our approach on image classification with a remote sensing data set that contains multiple label votes by domain experts for each image: The incorporation of label uncertainty helps the model to generalize better to unseen data and increases model performance. Similar to existing calibration methods, the distributional labels lead to better-calibrated probabilities, which in turn yield more certain and trustworthy predictions. △ Less

Submitted 30 May, 2022; originally announced May 2022.

Exponential Random Graph Models for Dynamic Signed Networks: An Application to International Relations

Authors: Cornelius Fritz, Marius Mehrl, Paul W. Thurner, Göran kauermann

Abstract: Substantive research in the Social Sciences regularly investigates signed networks, where edges between actors are either positive or negative. For instance, schoolchildren can be friends or rivals, just as countries can cooperate or fight each other. This research often builds on structural balance theory, one of the earliest and most prominent network theories, making signed networks one of the… ▽ More Substantive research in the Social Sciences regularly investigates signed networks, where edges between actors are either positive or negative. For instance, schoolchildren can be friends or rivals, just as countries can cooperate or fight each other. This research often builds on structural balance theory, one of the earliest and most prominent network theories, making signed networks one of the most frequently studied matters in social network analysis. While the theorization and description of signed networks have thus made significant progress, the inferential study of tie formation within them remains limited in the absence of appropriate statistical models. In this paper we fill this gap by proposing the Signed Exponential Random Graph Model (SERGM), extending the well-known Exponential Random Graph Model (ERGM) to networks where ties are not binary but negative or positive if a tie exists. Since most networks are dynamically evolving systems, we specify the model for both cross-sectional and dynamic networks. Based on structural hypotheses derived from structural balance theory, we formulate interpretable signed network statistics, capturing dynamics such as "the enemy of my enemy is my friend". In our empirical application, we use the SERGM to analyze cooperation and conflict between countries within the international state system. △ Less

Submitted 24 May, 2022; originally announced May 2022.

Tempus Volat, Hora Fugit -- A Survey of Tie-Oriented Dynamic Network Models in Discrete and Continuous Time

Authors: Cornelius Fritz, Michael Lebacher, Göran Kauermann

Abstract: Given the growing number of available tools for modeling dynamic networks, the choice of a suitable model becomes central. The goal of this survey is to provide an overview of tie-oriented dynamic network models. The survey is focused on introducing binary network models with their corresponding assumptions, advantages, and shortfalls. The models are divided according to generating processes, oper… ▽ More Given the growing number of available tools for modeling dynamic networks, the choice of a suitable model becomes central. The goal of this survey is to provide an overview of tie-oriented dynamic network models. The survey is focused on introducing binary network models with their corresponding assumptions, advantages, and shortfalls. The models are divided according to generating processes, operating in discrete and continuous time. First, we introduce the Temporal Exponential Random Graph Model (TERGM) and the Separable TERGM (STERGM), both being time-discrete models. These models are then contrasted with continuous process models, focusing on the Relational Event Model (REM). We additionally show how the REM can handle time-clustered observations, i.e., continuous time data observed at discrete time points. Besides the discussion of theoretical properties and fitting procedures, we specifically focus on the application of the models on two networks that represent international arms transfers and email exchange. The data allow to demonstrate the applicability and interpretation of the network models. △ Less

Submitted 28 August, 2019; v1 submitted 23 May, 2019; originally announced May 2019.