-
MirrorCheck: Efficient Adversarial Defense for Vision-Language Models
Authors:
Samar Fares,
Klea Ziu,
Toluwani Aremu,
Nikita Durasov,
Martin Takáč,
Pascal Fua,
Karthik Nandakumar,
Ivan Laptev
Abstract:
Vision-Language Models (VLMs) are becoming increasingly vulnerable to adversarial attacks as various novel attack strategies are being proposed against these models. While existing defenses excel in unimodal contexts, they currently fall short in safeguarding VLMs against adversarial threats. To mitigate this vulnerability, we propose a novel, yet elegantly simple approach for detecting adversaria…
▽ More
Vision-Language Models (VLMs) are becoming increasingly vulnerable to adversarial attacks as various novel attack strategies are being proposed against these models. While existing defenses excel in unimodal contexts, they currently fall short in safeguarding VLMs against adversarial threats. To mitigate this vulnerability, we propose a novel, yet elegantly simple approach for detecting adversarial samples in VLMs. Our method leverages Text-to-Image (T2I) models to generate images based on captions produced by target VLMs. Subsequently, we calculate the similarities of the embeddings of both input and generated images in the feature space to identify adversarial samples. Empirical evaluations conducted on different datasets validate the efficacy of our approach, outperforming baseline methods adapted from image classification domains. Furthermore, we extend our methodology to classification tasks, showcasing its adaptability and model-agnostic nature. Theoretical analyses and empirical findings also show the resilience of our approach against adaptive attacks, positioning it as an excellent defense mechanism for real-world deployment against adversarial threats.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
Redefining Contributions: Shapley-Driven Federated Learning
Authors:
Nurbek Tastan,
Samar Fares,
Toluwani Aremu,
Samuel Horvath,
Karthik Nandakumar
Abstract:
Federated learning (FL) has emerged as a pivotal approach in machine learning, enabling multiple participants to collaboratively train a global model without sharing raw data. While FL finds applications in various domains such as healthcare and finance, it is challenging to ensure global model convergence when participants do not contribute equally and/or honestly. To overcome this challenge, pri…
▽ More
Federated learning (FL) has emerged as a pivotal approach in machine learning, enabling multiple participants to collaboratively train a global model without sharing raw data. While FL finds applications in various domains such as healthcare and finance, it is challenging to ensure global model convergence when participants do not contribute equally and/or honestly. To overcome this challenge, principled mechanisms are required to evaluate the contributions made by individual participants in the FL setting. Existing solutions for contribution assessment rely on general accuracy evaluation, often failing to capture nuanced dynamics and class-specific influences. This paper proposes a novel contribution assessment method called ShapFed for fine-grained evaluation of participant contributions in FL. Our approach uses Shapley values from cooperative game theory to provide a granular understanding of class-specific influences. Based on ShapFed, we introduce a weighted aggregation method called ShapFed-WA, which outperforms conventional federated averaging, especially in class-imbalanced scenarios. Personalizing participant updates based on their contributions further enhances collaborative fairness by delivering differentiated models commensurate with the participant contributions. Experiments on CIFAR-10, Chest X-Ray, and Fed-ISIC2019 datasets demonstrate the effectiveness of our approach in improving utility, efficiency, and fairness in FL systems. The code can be found at https://github.com/tnurbek/shapfed.
△ Less
Submitted 1 June, 2024;
originally announced June 2024.
-
Regularized PolyKervNets: Optimizing Expressiveness and Efficiency for Private Inference in Deep Neural Networks
Authors:
Toluwani Aremu
Abstract:
Private computation of nonlinear functions, such as Rectified Linear Units (ReLUs) and max-pooling operations, in deep neural networks (DNNs) poses significant challenges in terms of storage, bandwidth, and time consumption. To address these challenges, there has been a growing interest in utilizing privacy-preserving techniques that leverage polynomial activation functions and kernelized convolut…
▽ More
Private computation of nonlinear functions, such as Rectified Linear Units (ReLUs) and max-pooling operations, in deep neural networks (DNNs) poses significant challenges in terms of storage, bandwidth, and time consumption. To address these challenges, there has been a growing interest in utilizing privacy-preserving techniques that leverage polynomial activation functions and kernelized convolutions as alternatives to traditional ReLUs. However, these alternative approaches often suffer from a trade-off between achieving faster private inference (PI) and sacrificing model accuracy. In particular, when applied to much deeper networks, these methods encounter training instabilities, leading to issues like exploding gradients (resulting in NaNs) or suboptimal approximations. In this study, we focus on PolyKervNets, a technique known for offering improved dynamic approximations in smaller networks but still facing instabilities in larger and more complex networks. Our primary objective is to empirically explore optimization-based training recipes to enhance the performance of PolyKervNets in larger networks. By doing so, we aim to potentially eliminate the need for traditional nonlinear activation functions, thereby advancing the state-of-the-art in privacy-preserving deep neural network architectures. Code can be found on GitHub at: \url{https://github.com/tolusophy/PolyKervNets/}
△ Less
Submitted 23 December, 2023;
originally announced December 2023.
-
Defending against cybersecurity threats to the payments and banking system
Authors:
Williams Haruna,
Toyin Ajiboro Aremu,
Yetunde Ajao Modupe
Abstract:
Cyber security threats to the payment and banking system have become a worldwide menace. The phenomenon has forced financial institutions to take risks as part of their business model. Hence, deliberate investment in sophisticated technologies and security measures has become imperative to safeguard against heavy financial losses and information breaches that may occur due to cyber-attacks. The pr…
▽ More
Cyber security threats to the payment and banking system have become a worldwide menace. The phenomenon has forced financial institutions to take risks as part of their business model. Hence, deliberate investment in sophisticated technologies and security measures has become imperative to safeguard against heavy financial losses and information breaches that may occur due to cyber-attacks. The proliferation of cyber crimes is a huge concern for various stakeholders in the banking sector. Usually, cyber-attacks are carried out via software systems running on a computing system in cyberspace. As such, to prevent risks of cyber-attacks on software systems, entities operating within cyberspace must be identified and the threats to the application security isolated after analyzing the vulnerabilities and develo** defense mechanisms. This paper will examine various approaches that identify assets in cyberspace, classify the cyber threats, provide security defenses and map security measures to control types and functionalities. Thus, adopting the right application to the security threats and defenses will aid IT professionals and users alike in making decisions for develo** a strong defense-in-depth mechanism.
△ Less
Submitted 15 December, 2022;
originally announced December 2022.
-
SSIVD-Net: A Novel Salient Super Image Classification & Detection Technique for Weaponized Violence
Authors:
Toluwani Aremu,
Li Zhiyuan,
Reem Alameeri,
Mustaqeem Khan,
Abdulmotaleb El Saddik
Abstract:
Detection of violence and weaponized violence in closed-circuit television (CCTV) footage requires a comprehensive approach. In this work, we introduce the \emph{Smart-City CCTV Violence Detection (SCVD)} dataset, specifically designed to facilitate the learning of weapon distribution in surveillance videos. To tackle the complexities of analyzing 3D surveillance video for violence recognition tas…
▽ More
Detection of violence and weaponized violence in closed-circuit television (CCTV) footage requires a comprehensive approach. In this work, we introduce the \emph{Smart-City CCTV Violence Detection (SCVD)} dataset, specifically designed to facilitate the learning of weapon distribution in surveillance videos. To tackle the complexities of analyzing 3D surveillance video for violence recognition tasks, we propose a novel technique called \emph{SSIVD-Net} (\textbf{S}alient-\textbf{S}uper-\textbf{I}mage for \textbf{V}iolence \textbf{D}etection). Our method reduces 3D video data complexity, dimensionality, and information loss while improving inference, performance, and explainability through salient-super-Image representations. Considering the scalability and sustainability requirements of futuristic smart cities, the authors introduce the \emph{Salient-Classifier}, a novel architecture combining a kernelized approach with a residual learning strategy. We evaluate variations of SSIVD-Net and Salient Classifier on our SCVD dataset and benchmark against state-of-the-art (SOTA) models commonly employed in violence detection. Our approach exhibits significant improvements in detecting both weaponized and non-weaponized violence instances. By advancing the SOTA in violence detection, our work offers a practical and scalable solution suitable for real-world applications. The proposed methodology not only addresses the challenges of violence detection in CCTV footage but also contributes to the understanding of weapon distribution in smart surveillance. Ultimately, our research findings should enable smarter and more secure cities, as well as enhance public safety measures.
△ Less
Submitted 7 November, 2023; v1 submitted 26 July, 2022;
originally announced July 2022.
