On the Value of Labeled Data and Symbolic Methods for Hidden Neuron Activation Analysis
Authors:
Abhilekha Dalal,
Rushrukh Rayan,
Adrita Barua,
Eugene Y. Vasserman,
Md Kamruzzaman Sarker,
Pascal Hitzler
Abstract:
A major challenge in Explainable AI is in correctly interpreting activations of hidden neurons: accurate interpretations would help answer the question of what a deep learning system internally detects as relevant in the input, demystifying the otherwise black-box nature of deep learning systems. The state of the art indicates that hidden node activations can, in some cases, be interpretable in a…
▽ More
A major challenge in Explainable AI is in correctly interpreting activations of hidden neurons: accurate interpretations would help answer the question of what a deep learning system internally detects as relevant in the input, demystifying the otherwise black-box nature of deep learning systems. The state of the art indicates that hidden node activations can, in some cases, be interpretable in a way that makes sense to humans, but systematic automated methods that would be able to hypothesize and verify interpretations of hidden neuron activations are underexplored. This is particularly the case for approaches that can both draw explanations from substantial background knowledge, and that are based on inherently explainable (symbolic) methods.
In this paper, we introduce a novel model-agnostic post-hoc Explainable AI method demonstrating that it provides meaningful interpretations. Our approach is based on using a Wikipedia-derived concept hierarchy with approximately 2 million classes as background knowledge, and utilizes OWL-reasoning-based Concept Induction for explanation generation. Additionally, we explore and compare the capabilities of off-the-shelf pre-trained multimodal-based explainable methods.
Our results indicate that our approach can automatically attach meaningful class expressions as explanations to individual neurons in the dense layer of a Convolutional Neural Network. Evaluation through statistical analysis and degree of concept activation in the hidden layer show that our method provides a competitive edge in both quantitative and qualitative aspects compared to prior work.
△ Less
Submitted 21 April, 2024;
originally announced April 2024.
Review of the Security of Backward-Compatible Automotive Inter-ECU Communication
Authors:
Chandra Sharma,
Samuel Moylan,
George Amariucai,
Eugene Y. Vasserman
Abstract:
Advanced electronic units inside modern vehicles have enhanced the driving experience, but also introduced a myriad of security problems due to the inherent limitations of the internal communication protocol. In the last two decades, a number of security threats have been identified and accordingly, security measures have been proposed. While a large body of research on the vehicular security doma…
▽ More
Advanced electronic units inside modern vehicles have enhanced the driving experience, but also introduced a myriad of security problems due to the inherent limitations of the internal communication protocol. In the last two decades, a number of security threats have been identified and accordingly, security measures have been proposed. While a large body of research on the vehicular security domain is focused on exposing vulnerabilities and proposing counter measures, there is an apparent paucity of research aimed at reviewing existing works on automotive security and at extracting insights. This paper provides a systematic review of security threats and countermeasures for the ubiquitous CAN bus communication protocol. It further exposes the limitations of the existing security measures, and discusses a seemingly-overlooked, simple, cost-effective and incrementally deployable solution which can provide a reasonable defense against a major class of packet injection attacks and many denial of service attacks.
△ Less
Submitted 26 September, 2021; v1 submitted 9 October, 2019;
originally announced October 2019.
