-
Exploring the Interplay of Interpretability and Robustness in Deep Neural Networks: A Saliency-guided Approach
Authors:
Amira Guesmi,
Nishant Suresh Aswani,
Muhammad Shafique
Abstract:
Adversarial attacks pose a significant challenge to deploying deep learning models in safety-critical applications. Maintaining model robustness while ensuring interpretability is vital for fostering trust and comprehension in these models. This study investigates the impact of Saliency-guided Training (SGT) on model robustness, a technique aimed at improving the clarity of saliency maps to deepen…
▽ More
Adversarial attacks pose a significant challenge to deploying deep learning models in safety-critical applications. Maintaining model robustness while ensuring interpretability is vital for fostering trust and comprehension in these models. This study investigates the impact of Saliency-guided Training (SGT) on model robustness, a technique aimed at improving the clarity of saliency maps to deepen understanding of the model's decision-making process. Experiments were conducted on standard benchmark datasets using various deep learning architectures trained with and without SGT. Findings demonstrate that SGT enhances both model robustness and interpretability. Additionally, we propose a novel approach combining SGT with standard adversarial training to achieve even greater robustness while preserving saliency map quality. Our strategy is grounded in the assumption that preserving salient features crucial for correctly classifying adversarial examples enhances model robustness, while masking non-relevant features improves interpretability. Our technique yields significant gains, achieving a 35\% and 20\% improvement in robustness against PGD attack with noise magnitudes of $0.2$ and $0.02$ for the MNIST and CIFAR-10 datasets, respectively, while producing high-quality saliency maps.
△ Less
Submitted 10 May, 2024;
originally announced May 2024.
-
Examining Changes in Internal Representations of Continual Learning Models Through Tensor Decomposition
Authors:
Nishant Suresh Aswani,
Amira Guesmi,
Muhammad Abdullah Hanif,
Muhammad Shafique
Abstract:
Continual learning (CL) has spurred the development of several methods aimed at consolidating previous knowledge across sequential learning. Yet, the evaluations of these methods have primarily focused on the final output, such as changes in the accuracy of predicted classes, overlooking the issue of representational forgetting within the model. In this paper, we propose a novel representation-bas…
▽ More
Continual learning (CL) has spurred the development of several methods aimed at consolidating previous knowledge across sequential learning. Yet, the evaluations of these methods have primarily focused on the final output, such as changes in the accuracy of predicted classes, overlooking the issue of representational forgetting within the model. In this paper, we propose a novel representation-based evaluation framework for CL models. This approach involves gathering internal representations from throughout the continual learning process and formulating three-dimensional tensors. The tensors are formed by stacking representations, such as layer activations, generated from several inputs and model `snapshots', throughout the learning process. By conducting tensor component analysis (TCA), we aim to uncover meaningful patterns about how the internal representations evolve, expecting to highlight the merits or shortcomings of examined CL strategies. We conduct our analyses across different model architectures and importance-based continual learning strategies, with a curated task selection. While the results of our approach mirror the difference in performance of various CL strategies, we found that our methodology did not directly highlight specialized clusters of neurons, nor provide an immediate understanding the evolution of filters. We believe a scaled down version of our approach will provide insight into the benefits and pitfalls of using TCA to study continual learning dynamics.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
HACK3D: Crowdsourcing the Assessment of Cybersecurity in Digital Manufacturing
Authors:
Michael Linares,
Nishant Aswani,
Gary Mac,
Chenglu **,
Fei Chen,
Nikhil Gupta,
Ramesh Karri
Abstract:
Digital manufacturing (DM) cyber-physical system is vulnerable to both cyber and physical attacks. HACK3D is a series of crowdsourcing red-team-blue-team events hosted by the NYU Center for Cybersecurity to assess the strength of the security methods embedded in designs using DM. This study summarizes the lessons learned from the past three offerings of HACK3D, including ingenious ways in which sk…
▽ More
Digital manufacturing (DM) cyber-physical system is vulnerable to both cyber and physical attacks. HACK3D is a series of crowdsourcing red-team-blue-team events hosted by the NYU Center for Cybersecurity to assess the strength of the security methods embedded in designs using DM. This study summarizes the lessons learned from the past three offerings of HACK3D, including ingenious ways in which skilled engineers can launch surprising attacks on DM designs not anticipated before. A key outcome is a taxonomy-guided creation of DM security benchmarks for use by the DM community.
△ Less
Submitted 16 April, 2021; v1 submitted 9 May, 2020;
originally announced May 2020.
-
Fuzzy Keyword Search over Encrypted Data using Symbol-Based Trie-traverse Search Scheme in Cloud Computing
Authors:
P. Naga Aswani,
K. Chandra Shekar
Abstract:
We exploit edit distance to quantify keywords similarity and develop two advanced techniques on constructing fuzzy keyword sets, which achieve optimized storage and representation overheads. We further propose a brand new symbol-based trie-traverse searching scheme, where a multi-way tree structure is built up using symbols transformed from the resulted fuzzy keyword sets. Through rigorous securit…
▽ More
We exploit edit distance to quantify keywords similarity and develop two advanced techniques on constructing fuzzy keyword sets, which achieve optimized storage and representation overheads. We further propose a brand new symbol-based trie-traverse searching scheme, where a multi-way tree structure is built up using symbols transformed from the resulted fuzzy keyword sets. Through rigorous security analysis, we show that our proposed solution is secure and privacy-preserving, while correctly realizing the goal of fuzzy keyword search. Extensive experimental results demonstrate the efficiency of the proposed solution.
△ Less
Submitted 7 November, 2012;
originally announced November 2012.
