-
Exploring the Potential of the Large Language Models (LLMs) in Identifying Misleading News Headlines
Authors:
Md Main Uddin Rony,
Md Mahfuzul Haque,
Mohammad Ali,
Ahmed Shatil Alam,
Naeemul Hassan
Abstract:
In the digital age, the prevalence of misleading news headlines poses a significant challenge to information integrity, necessitating robust detection mechanisms. This study explores the efficacy of Large Language Models (LLMs) in identifying misleading versus non-misleading news headlines. Utilizing a dataset of 60 articles, sourced from both reputable and questionable outlets across health, scie…
▽ More
In the digital age, the prevalence of misleading news headlines poses a significant challenge to information integrity, necessitating robust detection mechanisms. This study explores the efficacy of Large Language Models (LLMs) in identifying misleading versus non-misleading news headlines. Utilizing a dataset of 60 articles, sourced from both reputable and questionable outlets across health, science & tech, and business domains, we employ three LLMs- ChatGPT-3.5, ChatGPT-4, and Gemini-for classification. Our analysis reveals significant variance in model performance, with ChatGPT-4 demonstrating superior accuracy, especially in cases with unanimous annotator agreement on misleading headlines. The study emphasizes the importance of human-centered evaluation in develo** LLMs that can navigate the complexities of misinformation detection, aligning technical proficiency with nuanced human judgment. Our findings contribute to the discourse on AI ethics, emphasizing the need for models that are not only technically advanced but also ethically aligned and sensitive to the subtleties of human interpretation.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
Examining the Role of Clickbait Headlines to Engage Readers with Reliable Health-related Information
Authors:
Sima Bhowmik,
Md Main Uddin Rony,
Md Mahfuzul Haque,
Kristen Alley Swain,
Naeemul Hassan
Abstract:
Clickbait headlines are frequently used to attract readers to read articles. Although this headline type has turned out to be a technique to engage readers with misleading items, it is still unknown whether the technique can be used to attract readers to reliable pieces. This study takes the opportunity to test its efficacy to engage readers with reliable health articles. A set of online surveys w…
▽ More
Clickbait headlines are frequently used to attract readers to read articles. Although this headline type has turned out to be a technique to engage readers with misleading items, it is still unknown whether the technique can be used to attract readers to reliable pieces. This study takes the opportunity to test its efficacy to engage readers with reliable health articles. A set of online surveys would be conducted to test readers' engagement with and perception about clickbait headlines with reliable articles. After that, we would design an automation system to generate clickabit headlines to maximize user engagement.
△ Less
Submitted 25 November, 2019;
originally announced November 2019.
-
Differences between Health Related News Articles from Reliable and Unreliable Media
Authors:
Sameer Dhoju,
Md Main Uddin Rony,
Naeemul Hassan
Abstract:
In this study, we examine a collection of health-related news articles published by reliable and unreliable media outlets. Our analysis shows that there are structural, topical, and semantic differences in the way reliable and unreliable media outlets conduct health journalism. We argue that the findings from this study will be useful for combating health disinformation problem.
In this study, we examine a collection of health-related news articles published by reliable and unreliable media outlets. Our analysis shows that there are structural, topical, and semantic differences in the way reliable and unreliable media outlets conduct health journalism. We argue that the findings from this study will be useful for combating health disinformation problem.
△ Less
Submitted 5 November, 2018;
originally announced November 2018.
-
Fact-checking Initiatives in Bangladesh, India, and Nepal: A Study of User Engagement and Challenges
Authors:
Md Mahfuzul Haque,
Mohammad Yousuf,
Zahedur Arman,
Md Main Uddin Rony,
Ahmed Shatil Alam,
Kazi Mehedi Hasan,
Md Khadimul Islam,
Naeemul Hassan
Abstract:
Fake news and misinformation spread in develo** countries as fast as they do in developed countries with increasing penetration of the internet and social media. However, fighting misinformation is more difficult in develo** countries where resources and necessary technologies are scarce. This study provides an understanding of the challenges various fact-checking initiatives face in three Sou…
▽ More
Fake news and misinformation spread in develo** countries as fast as they do in developed countries with increasing penetration of the internet and social media. However, fighting misinformation is more difficult in develo** countries where resources and necessary technologies are scarce. This study provides an understanding of the challenges various fact-checking initiatives face in three South Asian countries--Bangladesh, India, and Nepal. In-depth interviews were conducted with senior editors of six fact-checking initiatives. Challenges identified include lack of resources, technologies, and political pressure. An analysis of Facebook pages of these initiatives shows increasing user engagement with their posts.
△ Less
Submitted 5 November, 2018;
originally announced November 2018.
-
A Large-scale Study of Social Media Sources in News Articles
Authors:
Md Main Uddin Rony,
Mohammad Yousuf,
Naeemul Hassan
Abstract:
In this study, we closely look at the use of social media contents as source or reference in the U.S. news media. Specifically, we examine about 60 thousand news articles published within the 5 years period of 2013-2017 by 153 U.S. media outlets and analyze use of social media content as source compared to other sources. We designed a social media source extraction algorithm and investigated the e…
▽ More
In this study, we closely look at the use of social media contents as source or reference in the U.S. news media. Specifically, we examine about 60 thousand news articles published within the 5 years period of 2013-2017 by 153 U.S. media outlets and analyze use of social media content as source compared to other sources. We designed a social media source extraction algorithm and investigated the extent and nature of social media source usage across different news topics. Our results show that uses of social media content in news almost doubled in five years. Unreliable media outlets rely on social media more than the mainstream media. Both mainstream and unreliable sites prefer Twitter to Facebook as a source of information.
△ Less
Submitted 30 October, 2018;
originally announced October 2018.
-
Diving Deep into Clickbaits: Who Use Them to What Extents in Which Topics with What Effects?
Authors:
Md Main Uddin Rony,
Naeemul Hassan,
Mohammad Yousuf
Abstract:
The use of alluring headlines (clickbait) to tempt the readers has become a growing practice nowadays. For the sake of existence in the highly competitive media industry, most of the on-line media including the mainstream ones, have started following this practice. Although the wide-spread practice of clickbait makes the reader's reliability on media vulnerable, a large scale analysis to reveal th…
▽ More
The use of alluring headlines (clickbait) to tempt the readers has become a growing practice nowadays. For the sake of existence in the highly competitive media industry, most of the on-line media including the mainstream ones, have started following this practice. Although the wide-spread practice of clickbait makes the reader's reliability on media vulnerable, a large scale analysis to reveal this fact is still absent. In this paper, we analyze 1.67 million Facebook posts created by 153 media organizations to understand the extent of clickbait practice, its impact and user engagement by using our own developed clickbait detection model. The model uses distributed sub-word embeddings learned from a large corpus. The accuracy of the model is 98.3%. Powered with this model, we further study the distribution of topics in clickbait and non-clickbait contents.
△ Less
Submitted 28 March, 2017;
originally announced March 2017.
