FACTIFY3M: A Benchmark for Multimodal Fact Verification with Explainability through 5W Question-Answering
Authors:
Megha Chakraborty,
Khushbu Pahwa,
Anku Rani,
Shreyas Chatterjee,
Dwip Dalal,
Harshit Dave,
Ritvik G,
Preethi Gurumurthy,
Adarsh Mahor,
Samahriti Mukherjee,
Aditya Pakala,
Ishan Paul,
Janvita Reddy,
Arghya Sarkar,
Kinjal Sensharma,
Aman Chadha,
Amit P. Sheth,
Amitava Das
Abstract:
Combating disinformation is one of the burning societal crises -- about 67% of the American population believes that disinformation produces a lot of uncertainty, and 10% of them knowingly propagate disinformation. Evidence shows that disinformation can manipulate democratic processes and public opinion, causing disruption in the share market, panic and anxiety in society, and even death during cr…
▽ More
Combating disinformation is one of the burning societal crises -- about 67% of the American population believes that disinformation produces a lot of uncertainty, and 10% of them knowingly propagate disinformation. Evidence shows that disinformation can manipulate democratic processes and public opinion, causing disruption in the share market, panic and anxiety in society, and even death during crises. Therefore, disinformation should be identified promptly and, if possible, mitigated. With approximately 3.2 billion images and 720,000 hours of video shared online daily on social media platforms, scalable detection of multimodal disinformation requires efficient fact verification. Despite progress in automatic text-based fact verification (e.g., FEVER, LIAR), the research community lacks substantial effort in multimodal fact verification. To address this gap, we introduce FACTIFY 3M, a dataset of 3 million samples that pushes the boundaries of the domain of fact verification via a multimodal fake news dataset, in addition to offering explainability through the concept of 5W question-answering. Salient features of the dataset include: (i) textual claims, (ii) ChatGPT-generated paraphrased claims, (iii) associated images, (iv) stable diffusion-generated additional images (i.e., visual paraphrases), (v) pixel-level image heatmap to foster image-text explainability of the claim, (vi) 5W QA pairs, and (vii) adversarial fake news stories.
△ Less
Submitted 30 October, 2023; v1 submitted 22 May, 2023;
originally announced June 2023.
Estimating dynamical parameters of two interacting galaxies using Deep Learning
Authors:
Adarsh Mahor,
Janvita Reddy,
Amitesh Singh,
Shashwat Singh
Abstract:
The science behind galaxy interaction and mergers has a fundamental role and gives us an insight into galaxy formation and its evolution. Fluctuating angular momentum is responsible for extraordinary events like polar rings, tidal tails, and ripples. To study different phenomena related to galaxy interactions, various parameters like the mass ratio of the interacting galaxy, orbital parameters, ma…
▽ More
The science behind galaxy interaction and mergers has a fundamental role and gives us an insight into galaxy formation and its evolution. Fluctuating angular momentum is responsible for extraordinary events like polar rings, tidal tails, and ripples. To study different phenomena related to galaxy interactions, various parameters like the mass ratio of the interacting galaxy, orbital parameters, mass distribution, morphologies are required. Convolutional Neural Networks (CNN) are widely used to classify image data. Thus, we used CNN as our approach to the problem. In this work, we will be using data from state-of-the-art magneto-hydrodynamic simulations of galaxy mergers from the GalMer database at different dynamical parameters using image snapshots of merging pairs of galaxies and feeding them to our Deep Learning model (ResNet). The dynamical parameters we are aiming for; would be spin, relative inclination ($i$), viewing angle ($θ$), and azimuthal angle ($φ$). We aim to download bulk data using the web scra** method. The first approach is to create different combinations of these parameters to form 60 classes. Feeding the data into the model, we achieved 93.63% accuracy. As we received good results in minute classification, we moved to our second approach, regression. Here the model can predict the continuous and exact values of the dynamical parameters. We have achieved a 99.86% R-squared value and the mean squared error of 0.0833 on testing data. In the end, we used data from Sloan Digital Sky Survey to test our trained model on some real images.
△ Less
Submitted 23 December, 2021;
originally announced December 2021.
