-
Real, fake and synthetic faces -- does the coin have three sides?
Authors:
Shahzeb Naeem,
Ramzi Al-Sharawi,
Muhammad Riyyan Khan,
Usman Tariq,
Abhinav Dhall,
Hasan Al-Nashash
Abstract:
With the ever-growing power of generative artificial intelligence, deepfake and artificially generated (synthetic) media have continued to spread online, which creates various ethical and moral concerns regarding their usage. To tackle this, we thus present a novel exploration of the trends and patterns observed in real, deepfake and synthetic facial images. The proposed analysis is done in two pa…
▽ More
With the ever-growing power of generative artificial intelligence, deepfake and artificially generated (synthetic) media have continued to spread online, which creates various ethical and moral concerns regarding their usage. To tackle this, we thus present a novel exploration of the trends and patterns observed in real, deepfake and synthetic facial images. The proposed analysis is done in two parts: firstly, we incorporate eight deep learning models and analyze their performances in distinguishing between the three classes of images. Next, we look to further delve into the similarities and differences between these three sets of images by investigating their image properties both in the context of the entire image as well as in the context of specific regions within the image. ANOVA test was also performed and provided further clarity amongst the patterns associated between the images of the three classes. From our findings, we observe that the investigated deeplearning models found it easier to detect synthetic facial images, with the ViT Patch-16 model performing best on this task with a class-averaged sensitivity, specificity, precision, and accuracy of 97.37%, 98.69%, 97.48%, and 98.25%, respectively. This observation was supported by further analysis of various image properties. We saw noticeable differences across the three category of images. This analysis can help us build better algorithms for facial image generation, and also shows that synthetic, deepfake and real face images are indeed three different classes.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
Generation and Detection of Sign Language Deepfakes -- A Linguistic and Visual Analysis
Authors:
Shahzeb Naeem,
Muhammad Riyyan Khan,
Usman Tariq,
Abhinav Dhall,
Carlos Ivan Colon,
Hasan Al-Nashash
Abstract:
A question in the realm of deepfakes is slowly emerging pertaining to whether we can go beyond facial deepfakes and whether it would be beneficial to society. Therefore, this research presents a positive application of deepfake technology in upper body generation, while performing sign-language for the Deaf and Hard of Hearing (DHoH) community. The resulting videos are later vetted with a sign lan…
▽ More
A question in the realm of deepfakes is slowly emerging pertaining to whether we can go beyond facial deepfakes and whether it would be beneficial to society. Therefore, this research presents a positive application of deepfake technology in upper body generation, while performing sign-language for the Deaf and Hard of Hearing (DHoH) community. The resulting videos are later vetted with a sign language expert. This is particularly helpful, given the intricate nature of sign language, a scarcity of sign language experts, and potential benefits for health and education. The objectives of this work encompass constructing a reliable deepfake dataset, evaluating its technical and visual credibility through computer vision and natural language processing models, and assessing the plausibility of the generated content. With over 1200 videos, featuring both previously seen and unseen individuals for the generation model, using the help of a sign language expert, we establish a deepfake dataset in sign language that can further be utilized to detect fake videos that may target certain people of determination.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
Compiling Recurrences over Dense and Sparse Arrays
Authors:
Shiv Sundram,
Muhammad Usman Tariq,
Fredrik Kjolstad
Abstract:
Recurrence equations lie at the heart of many computational paradigms including dynamic programming, graph analysis, and linear solvers. These equations are often expensive to compute and much work has gone into optimizing them for different situations. The set of recurrence implementations is a large design space across the set of all recurrences (e.g., the Viterbi and Floyd-Warshall algorithms),…
▽ More
Recurrence equations lie at the heart of many computational paradigms including dynamic programming, graph analysis, and linear solvers. These equations are often expensive to compute and much work has gone into optimizing them for different situations. The set of recurrence implementations is a large design space across the set of all recurrences (e.g., the Viterbi and Floyd-Warshall algorithms), the choice of data structures (e.g., dense and sparse matrices), and the set of different loop orders. Optimized library implementations do not exist for most points in this design space, and developers must therefore often manually implement and optimize recurrences. We present a general framework for compiling recurrence equations into native code corresponding to any valid point in this general design space. In this framework, users specify a system of recurrences, the type of data structures for storing the input and outputs, and a set of scheduling primitives for optimization. A greedy algorithm then takes this specification and lowers it into a native program that respects the dependencies inherent to the recurrence equation. We describe the compiler transformations necessary to lower this high-level specification into native parallel code for either sparse and dense data structures and provide an algorithm for determining whether the recurrence system is solvable with the provided scheduling primitives. We evaluate the performance and correctness of the generated code on various computational tasks from domains including dense and sparse matrix solvers, dynamic programming, graph problems, and sparse tensor algebra. We demonstrate that generated code has competitive performance to handwritten implementations in libraries.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Brain Tumor Synthetic Data Generation with Adaptive StyleGANs
Authors:
Usama Tariq,
Rizwan Qureshi,
Anas Zafar,
Danyal Aftab,
Jia Wu,
Tanvir Alam,
Zubair Shah,
Hazrat Ali
Abstract:
Generative models have been very successful over the years and have received significant attention for synthetic data generation. As deep learning models are getting more and more complex, they require large amounts of data to perform accurately. In medical image analysis, such generative models play a crucial role as the available data is limited due to challenges related to data privacy, lack of…
▽ More
Generative models have been very successful over the years and have received significant attention for synthetic data generation. As deep learning models are getting more and more complex, they require large amounts of data to perform accurately. In medical image analysis, such generative models play a crucial role as the available data is limited due to challenges related to data privacy, lack of data diversity, or uneven data distributions. In this paper, we present a method to generate brain tumor MRI images using generative adversarial networks. We have utilized StyleGAN2 with ADA methodology to generate high-quality brain MRI with tumors while using a significantly smaller amount of training data when compared to the existing approaches. We use three pre-trained models for transfer learning. Results demonstrate that the proposed method can learn the distributions of brain tumors. Furthermore, the model can generate high-quality synthetic brain MRI with a tumor that can limit the small sample size issues. The approach can addresses the limited data availability by generating realistic-looking brain MRI with tumors. The code is available at: ~\url{https://github.com/rizwanqureshi123/Brain-Tumor-Synthetic-Data}.
△ Less
Submitted 4 December, 2022;
originally announced December 2022.
-
Self-Supervised Approach for Facial Movement Based Optical Flow
Authors:
Muhannad Alkaddour,
Usman Tariq,
Abhinav Dhall
Abstract:
Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. The aim of…
▽ More
Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. The aim of this work is threefold: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train Convolutional Neural Networks (CNN) for predicting optical flow; and (3) using the learned optical flow in micro-expression recognition to demonstrate its effectiveness. We generate optical flow ground truth using facial key-points in the BP4D-Spontaneous dataset. The generated optical flow is used to train the FlowNetS architecture to test its performance on the generated dataset. The performance of FlowNetS trained on face images surpassed that of other optical flow CNN architectures, demonstrating its usefulness. Our optical flow features are further compared with other methods using the STSTNet micro-expression classifier, and the results indicate that the optical flow obtained using this work has promising applications in facial expression analysis.
△ Less
Submitted 3 May, 2021;
originally announced May 2021.
-
Hyperrealistic Image Inpainting with Hypergraphs
Authors:
Gourav Wadhwa,
Abhinav Dhall,
Subrahmanyam Murala,
Usman Tariq
Abstract:
Image inpainting is a non-trivial task in computer vision due to multiple possibilities for filling the missing data, which may be dependent on the global information of the image. Most of the existing approaches use the attention mechanism to learn the global context of the image. This attention mechanism produces semantically plausible but blurry results because of incapability to capture the gl…
▽ More
Image inpainting is a non-trivial task in computer vision due to multiple possibilities for filling the missing data, which may be dependent on the global information of the image. Most of the existing approaches use the attention mechanism to learn the global context of the image. This attention mechanism produces semantically plausible but blurry results because of incapability to capture the global context. In this paper, we introduce hypergraph convolution on spatial features to learn the complex relationship among the data. We introduce a trainable mechanism to connect nodes using hyperedges for hypergraph convolution. To the best of our knowledge, hypergraph convolution have never been used on spatial features for any image-to-image tasks in computer vision. Further, we introduce gated convolution in the discriminator to enforce local consistency in the predicted image. The experiments on Places2, CelebA-HQ, Paris Street View, and Facades datasets, show that our approach achieves state-of-the-art results.
△ Less
Submitted 5 November, 2020;
originally announced November 2020.
-
Mobile node localization in cellular networks
Authors:
Yasir Malik,
Kishwer Abdul Khaliq,
Bessam Abdulrazak,
Usman Tariq
Abstract:
Location information is the major component in location based applications. This information is used in different safety and service oriented applications to provide users with services according to their Geolocation. There are many approaches to locate mobile nodes in indoor and outdoor environments. In this paper, we are interested in outdoor localization particularly in cellular networks of mob…
▽ More
Location information is the major component in location based applications. This information is used in different safety and service oriented applications to provide users with services according to their Geolocation. There are many approaches to locate mobile nodes in indoor and outdoor environments. In this paper, we are interested in outdoor localization particularly in cellular networks of mobile nodes and presented a localization method based on cell and user location information. Our localization method is based on hello message delay (sending and receiving time) and coordinate information of Base Transceiver Station (BTSs). To validate our method across cellular network, we implemented and simulated our method in two scenarios i.e. maintaining database of base stations in centralize and distributed system. Simulation results show the effectiveness of our approach and its implementation applicability in telecommunication systems.
△ Less
Submitted 10 January, 2012;
originally announced January 2012.
-
NACS: non-overlap** AP's caching scheme to reduce handoff in 802.11 wireless LAN
Authors:
Usman Tariq,
Yasir Malik,
Man-Pyo Hong
Abstract:
With the escalation of the IEEE 802.11 based wireless networks, voice over IP and analogous applications are also used over wireless networks. Recently, the wireless LAN systems are spaciously deployed for public Internet services. In public wireless LAN systems, reliable user authentication and mobility support are indispensable issues. When a mobile device budges out the range of one access poin…
▽ More
With the escalation of the IEEE 802.11 based wireless networks, voice over IP and analogous applications are also used over wireless networks. Recently, the wireless LAN systems are spaciously deployed for public Internet services. In public wireless LAN systems, reliable user authentication and mobility support are indispensable issues. When a mobile device budges out the range of one access point (AP) and endeavor to connect to new AP, it performs handoff. Contemporarily, PNC and SNC were proposed to propagate the MN context to the entire neighboring AP's on the wireless network with the help of neighbor graph. In this paper, we proposed a non-overlap** AP's caching scheme (NACS), which propagates the mobile node context to those AP's which do not overlap with the current AP. To capture the topology of non-overlap** AP's in the wireless network, non-overlap** graph (NOG) is generated at each AP. Simulation results shows that NACS reduces the signaling cost of propagating the MN context to the neighbor AP's in the wireless network.
△ Less
Submitted 20 December, 2011;
originally announced December 2011.
