-
Hostile Counterspeech Drives Users From Hate Subreddits
Authors:
Daniel Hickey,
Matheus Schmitz,
Daniel M. T. Fessler,
Paul E. Smaldino,
Kristina Lerman,
Goran Murić,
Keith Burghardt
Abstract:
Counterspeech -- speech that opposes hate speech -- has gained significant attention recently as a strategy to reduce hate on social media. While previous studies suggest that counterspeech can somewhat reduce hate speech, little is known about its effects on participation in online hate communities, nor which counterspeech tactics reduce harmful behavior. We begin to address these gaps by identif…
▽ More
Counterspeech -- speech that opposes hate speech -- has gained significant attention recently as a strategy to reduce hate on social media. While previous studies suggest that counterspeech can somewhat reduce hate speech, little is known about its effects on participation in online hate communities, nor which counterspeech tactics reduce harmful behavior. We begin to address these gaps by identifying 25 large hate communities ("subreddits") within Reddit and analyzing the effect of counterspeech on newcomers within these communities. We first construct a new public dataset of carefully annotated counterspeech and non-counterspeech comments within these subreddits. We use this dataset to train a state-of-the-art counterspeech detection model. Next, we use matching to evaluate the causal effects of hostile and non-hostile counterspeech on the engagement of newcomers in hate subreddits. We find that, while non-hostile counterspeech is ineffective at kee** users from fully disengaging from these hate subreddits, a single hostile counterspeech comment substantially reduces both future likelihood of engagement. While offering nuance to the understanding of counterspeech efficacy, these results a) leave unanswered the question of whether hostile counterspeech dissuades newcomers from participation in online hate writ large, or merely drives them into less-moderated and more extreme hate communities, and b) raises ethical considerations about hostile counterspeech, which is both comparatively common and might exacerbate rather than mitigate the net level of antagonism in society. These findings underscore the importance of future work to improve counterspeech tactics and minimize unintended harm.
△ Less
Submitted 28 May, 2024;
originally announced May 2024.
-
The Peripatetic Hater: Predicting Movement Among Hate Subreddits
Authors:
Daniel Hickey,
Daniel M. T. Fessler,
Kristina Lerman,
Keith Burghardt
Abstract:
Many online hate groups exist to disparage others based on race, gender identity, sex, or other characteristics. The accessibility of these communities allows users to join multiple types of hate groups (e.g., a racist community and misogynistic community), which calls into question whether these peripatetic users could be further radicalized compared to users that stay in one type of hate group.…
▽ More
Many online hate groups exist to disparage others based on race, gender identity, sex, or other characteristics. The accessibility of these communities allows users to join multiple types of hate groups (e.g., a racist community and misogynistic community), which calls into question whether these peripatetic users could be further radicalized compared to users that stay in one type of hate group. However, little is known about the dynamics of joining multiple types of hate groups, nor the effect of these groups on peripatetic users. In this paper, we develop a new method to classify hate subreddits, and the identities they disparage, which we use to better understand how users become peripatetic (join different types of hate subreddits). The hate classification technique utilizes human-validated LLMs to extract the protected identities attacked, if any, across 168 subreddits. We then cluster identity-attacking subreddits to discover three broad categories of hate: racist, anti-LGBTQ, and misogynistic. We show that becoming active in a user's first hate subreddit can cause them to become active in additional hate subreddits of a different category. We also find that users who join additional hate subreddits, especially of a different category, become more active in hate subreddits as a whole and develop a wider hate group lexicon. We are therefore motivated to train an AI model that we find usefully predicts the hate categories users will become active in based on post text read and written. The accuracy of this model may be partly driven by peripatetic users often using the language of hate subreddits they eventually join. Overall, these results highlight the unique risks associated with hate communities on a social media platform, as discussion of alternative targets of hate may lead users to target more protected identities.
△ Less
Submitted 27 May, 2024;
originally announced May 2024.
-
Auditing Elon Musk's Impact on Hate Speech and Bots
Authors:
Daniel Hickey,
Matheus Schmitz,
Daniel Fessler,
Paul Smaldino,
Goran Muric,
Keith Burghardt
Abstract:
On October 27th, 2022, Elon Musk purchased Twitter, becoming its new CEO and firing many top executives in the process. Musk listed fewer restrictions on content moderation and removal of spam bots among his goals for the platform. Given findings of prior research on moderation and hate speech in online communities, the promise of less strict content moderation poses the concern that hate will ris…
▽ More
On October 27th, 2022, Elon Musk purchased Twitter, becoming its new CEO and firing many top executives in the process. Musk listed fewer restrictions on content moderation and removal of spam bots among his goals for the platform. Given findings of prior research on moderation and hate speech in online communities, the promise of less strict content moderation poses the concern that hate will rise on Twitter. We examine the levels of hate speech and prevalence of bots before and after Musk's acquisition of the platform. We find that hate speech rose dramatically upon Musk purchasing Twitter and the prevalence of most types of bots increased, while the prevalence of astroturf bots decreased.
△ Less
Submitted 28 January, 2024; v1 submitted 8 April, 2023;
originally announced April 2023.
-
No Love Among Haters: Negative Interactions Reduce Hate Community Engagement
Authors:
Daniel Hickey,
Matheus Schmitz,
Daniel Fessler,
Paul Smaldino,
Goran Muric,
Keith Burghardt
Abstract:
While online hate groups pose significant risks to the health of online platforms and safety of marginalized groups, little is known about what causes users to become active in hate groups and the effect of social interactions on furthering their engagement. We address this gap by first develo** tools to find hate communities within Reddit, and then augment 11 subreddits extracted with 14 known…
▽ More
While online hate groups pose significant risks to the health of online platforms and safety of marginalized groups, little is known about what causes users to become active in hate groups and the effect of social interactions on furthering their engagement. We address this gap by first develo** tools to find hate communities within Reddit, and then augment 11 subreddits extracted with 14 known hateful subreddits (25 in total). Using causal inference methods, we evaluate the effect of replies on engagement in hateful subreddits by comparing users who receive replies to their first comment (the treatment) to equivalent control users who do not. We find users who receive replies are less likely to become engaged in hateful subreddits than users who do not, while the opposite effect is observed for a matched sample of similar-sized non-hateful subreddits. Using the Google Perspective API and VADER, we discover that hateful community first-repliers are more toxic, negative, and attack the posters more often than non-hateful first-repliers. In addition, we uncover a negative correlation between engagement and attacks or toxicity of first-repliers. We simulate the cumulative engagement of hateful and non-hateful subreddits under the contra-positive scenario of friendly first-replies, finding that attacks dramatically reduce engagement in hateful subreddits. These results counter-intuitively imply that, although under-moderated communities allow hate to fester, the resulting environment is such that direct social interaction does not encourage further participation, thus endogenously constraining the harmful role that these communities could play as recruitment venues for antisocial beliefs.
△ Less
Submitted 23 March, 2023;
originally announced March 2023.
