-
View-Invariant Pixelwise Anomaly Detection in Multi-object Scenes with Adaptive View Synthesis
Authors:
Subin Varghese,
Vedhus Hoskere
Abstract:
The inspection and monitoring of infrastructure assets typically requires identifying visual anomalies in scenes periodically photographed over time. Images collected manually or with robots such as unmanned aerial vehicles from the same scene at different instances in time are typically not perfectly aligned. Supervised segmentation methods can be applied to identify known problems, but unsupervi…
▽ More
The inspection and monitoring of infrastructure assets typically requires identifying visual anomalies in scenes periodically photographed over time. Images collected manually or with robots such as unmanned aerial vehicles from the same scene at different instances in time are typically not perfectly aligned. Supervised segmentation methods can be applied to identify known problems, but unsupervised anomaly detection approaches are required when unknown anomalies occur. Current unsupervised pixel-level anomaly detection methods have mainly been developed for industrial settings where the camera position is known and constant. However, we find that these methods fail to generalize to the case when images are not perfectly aligned. We term the problem of unsupervised anomaly detection between two such imperfectly aligned sets of images as Scene Anomaly Detection (Scene AD). We present a novel network termed OmniAD to address the Scene AD problem posed. Specifically, we refine the anomaly detection method reverse distillation to achieve a 40% increase in pixel-level anomaly detection performance. The network's performance is further demonstrated to improve with two new data augmentation strategies proposed that leverage novel view synthesis and camera localization to improve generalization. We validate our approach with qualitative and quantitative results on a new dataset, ToyCity, the first Scene AD dataset with multiple objects, as well as on the established single object-centric dataset, MAD. https://drags99.github.io/OmniAD/
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
The Model Openness Framework: Promoting Completeness and Openness for Reproducibility, Transparency, and Usability in Artificial Intelligence
Authors:
Matt White,
Ibrahim Haddad,
Cailean Osborne,
Xiao-Yang Liu Yanglet,
Ahmed Abdelmonsef,
Sachin Varghese
Abstract:
Generative AI (GAI) offers unprecedented opportunities for research and innovation, but its commercialization has raised concerns about transparency, reproducibility, and safety. Many open GAI models lack the necessary components for full understanding and reproducibility, and some use restrictive licenses whilst claiming to be ``open-source''. To address these concerns, we propose the Model Openn…
▽ More
Generative AI (GAI) offers unprecedented opportunities for research and innovation, but its commercialization has raised concerns about transparency, reproducibility, and safety. Many open GAI models lack the necessary components for full understanding and reproducibility, and some use restrictive licenses whilst claiming to be ``open-source''. To address these concerns, we propose the Model Openness Framework (MOF), a ranked classification system that rates machine learning models based on their completeness and openness, following principles of open science, open source, open data, and open access. The MOF requires specific components of the model development lifecycle to be included and released under appropriate open licenses. This framework aims to prevent misrepresentation of models claiming to be open, guide researchers and developers in providing all model components under permissive licenses, and help individuals and organizations identify models that can be safely adopted without restrictions. By promoting transparency and reproducibility, the MOF combats ``openwashing'' practices and establishes completeness and openness as primary criteria alongside the core tenets of responsible AI. Wide adoption of the MOF will foster a more open AI ecosystem, benefiting research, innovation, and adoption of state-of-the-art models.
△ Less
Submitted 3 June, 2024; v1 submitted 20 March, 2024;
originally announced March 2024.
-
Digital Twin-based Intrusion Detection for Industrial Control Systems
Authors:
Seba Anna Varghese,
Alireza Dehlaghi Ghadim,
Ali Balador,
Zahra Alimadadi,
Panos Papadimitratos
Abstract:
Digital twins have recently gained significant interest in simulation, optimization, and predictive maintenance of Industrial Control Systems (ICS). Recent studies discuss the possibility of using digital twins for intrusion detection in industrial systems. Accordingly, this study contributes to a digital twin-based security framework for industrial control systems, extending its capabilities for…
▽ More
Digital twins have recently gained significant interest in simulation, optimization, and predictive maintenance of Industrial Control Systems (ICS). Recent studies discuss the possibility of using digital twins for intrusion detection in industrial systems. Accordingly, this study contributes to a digital twin-based security framework for industrial control systems, extending its capabilities for simulation of attacks and defense mechanisms. Four types of process-aware attack scenarios are implemented on a standalone open-source digital twin of an industrial filling plant: command injection, network Denial of Service (DoS), calculated measurement modification, and naive measurement modification. A stacked ensemble classifier is proposed as the real-time intrusion detection, based on the offline evaluation of eight supervised machine learning algorithms. The designed stacked model outperforms previous methods in terms of F1-Score and accuracy, by combining the predictions of various algorithms, while it can detect and classify intrusions in near real-time (0.1 seconds). This study also discusses the practicality and benefits of the proposed digital twin-based security framework.
△ Less
Submitted 20 July, 2022;
originally announced July 2022.
-
Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
Authors:
Sebastian Houben,
Stephanie Abrecht,
Maram Akila,
Andreas Bär,
Felix Brockherde,
Patrick Feifel,
Tim Fingscheidt,
Sujan Sai Gannamaneni,
Seyed Eghbal Ghobadi,
Ahmed Hammam,
Anselm Haselhoff,
Felix Hauser,
Christian Heinzemann,
Marco Hoffmann,
Nikhil Kapoor,
Falk Kappel,
Marvin Klingner,
Jan Kronenberger,
Fabian Küppers,
Jonas Löhdefink,
Michael Mlynarski,
Michael Mock,
Firas Mualla,
Svetlana Pavlitskaya,
Maximilian Poretschkin
, et al. (16 additional authors not shown)
Abstract:
The use of deep neural networks (DNNs) in safety-critical applications like mobile health and autonomous driving is challenging due to numerous model-inherent shortcomings. These shortcomings are diverse and range from a lack of generalization over insufficient interpretability to problems with malicious inputs. Cyber-physical systems employing DNNs are therefore likely to suffer from safety conce…
▽ More
The use of deep neural networks (DNNs) in safety-critical applications like mobile health and autonomous driving is challenging due to numerous model-inherent shortcomings. These shortcomings are diverse and range from a lack of generalization over insufficient interpretability to problems with malicious inputs. Cyber-physical systems employing DNNs are therefore likely to suffer from safety concerns. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. This work provides a structured and broad overview of them. We first identify categories of insufficiencies to then describe research activities aiming at their detection, quantification, or mitigation. Our paper addresses both machine learning experts and safety engineers: The former ones might profit from the broad range of machine learning topics covered and discussions on limitations of recent methods. The latter ones might gain insights into the specifics of modern ML methods. We moreover hope that our contribution fuels discussions on desiderata for ML systems and strategies on how to propel existing approaches accordingly.
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
-
The Vulnerability of Semantic Segmentation Networks to Adversarial Attacks in Autonomous Driving: Enhancing Extensive Environment Sensing
Authors:
Andreas Bär,
Jonas Löhdefink,
Nikhil Kapoor,
Serin J. Varghese,
Fabian Hüger,
Peter Schlicht,
Tim Fingscheidt
Abstract:
Enabling autonomous driving (AD) can be considered one of the biggest challenges in today's technology. AD is a complex task accomplished by several functionalities, with environment perception being one of its core functions. Environment perception is usually performed by combining the semantic information captured by several sensors, i.e., lidar or camera. The semantic information from the respe…
▽ More
Enabling autonomous driving (AD) can be considered one of the biggest challenges in today's technology. AD is a complex task accomplished by several functionalities, with environment perception being one of its core functions. Environment perception is usually performed by combining the semantic information captured by several sensors, i.e., lidar or camera. The semantic information from the respective sensor can be extracted by using convolutional neural networks (CNNs) for dense prediction. In the past, CNNs constantly showed state-of-the-art performance on several vision-related tasks, such as semantic segmentation of traffic scenes using nothing but the red-green-blue (RGB) images provided by a camera. Although CNNs obtain state-of-the-art performance on clean images, almost imperceptible changes to the input, referred to as adversarial perturbations, may lead to fatal deception. The goal of this article is to illuminate the vulnerability aspects of CNNs used for semantic segmentation with respect to adversarial attacks, and share insights into some of the existing known adversarial defense strategies. We aim to clarify the advantages and disadvantages associated with applying CNNs for environment perception in AD to serve as a motivation for future research in this field.
△ Less
Submitted 13 January, 2021; v1 submitted 11 January, 2021;
originally announced January 2021.
-
Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimates
Authors:
Kira Maag,
Matthias Rottmann,
Serin Varghese,
Fabian Hueger,
Peter Schlicht,
Hanno Gottschalk
Abstract:
Instance segmentation with neural networks is an essential task in environment perception. In many works, it has been observed that neural networks can predict false positive instances with high confidence values and true positives with low ones. Thus, it is important to accurately model the uncertainties of neural networks in order to prevent safety issues and foster interpretability. In applicat…
▽ More
Instance segmentation with neural networks is an essential task in environment perception. In many works, it has been observed that neural networks can predict false positive instances with high confidence values and true positives with low ones. Thus, it is important to accurately model the uncertainties of neural networks in order to prevent safety issues and foster interpretability. In applications such as automated driving, the reliability of neural networks is of highest interest. In this paper, we present a time-dynamic approach to model uncertainties of instance segmentation networks and apply this to the detection of false positives as well as the estimation of prediction quality. The availability of image sequences in online applications allows for tracking instances over multiple frames. Based on an instances history of shape and uncertainty information, we construct temporal instance-wise aggregated metrics. The latter are used as input to post-processing models that estimate the prediction quality in terms of instance-wise intersection over union. The proposed method only requires a readily trained neural network (that may operate on single frames) and video sequence input. In our experiments, we further demonstrate the use of the proposed method by replacing the traditional score value from object detection and thereby improving the overall performance of the instance segmentation network.
△ Less
Submitted 13 April, 2021; v1 submitted 14 December, 2020;
originally announced December 2020.
-
From a Fourier-Domain Perspective on Adversarial Examples to a Wiener Filter Defense for Semantic Segmentation
Authors:
Nikhil Kapoor,
Andreas Bär,
Serin Varghese,
Jan David Schneider,
Fabian Hüger,
Peter Schlicht,
Tim Fingscheidt
Abstract:
Despite recent advancements, deep neural networks are not robust against adversarial perturbations. Many of the proposed adversarial defense approaches use computationally expensive training mechanisms that do not scale to complex real-world tasks such as semantic segmentation, and offer only marginal improvements. In addition, fundamental questions on the nature of adversarial perturbations and t…
▽ More
Despite recent advancements, deep neural networks are not robust against adversarial perturbations. Many of the proposed adversarial defense approaches use computationally expensive training mechanisms that do not scale to complex real-world tasks such as semantic segmentation, and offer only marginal improvements. In addition, fundamental questions on the nature of adversarial perturbations and their relation to the network architecture are largely understudied. In this work, we study the adversarial problem from a frequency domain perspective. More specifically, we analyze discrete Fourier transform (DFT) spectra of several adversarial images and report two major findings: First, there exists a strong connection between a model architecture and the nature of adversarial perturbations that can be observed and addressed in the frequency domain. Second, the observed frequency patterns are largely image- and attack-type independent, which is important for the practical impact of any defense making use of such patterns. Motivated by these findings, we additionally propose an adversarial defense method based on the well-known Wiener filters that captures and suppresses adversarial frequencies in a data-driven manner. Our proposed method not only generalizes across unseen attacks but also beats five existing state-of-the-art methods across two models in a variety of attack settings.
△ Less
Submitted 21 April, 2021; v1 submitted 2 December, 2020;
originally announced December 2020.
-
A Self-Supervised Feature Map Augmentation (FMA) Loss and Combined Augmentations Finetuning to Efficiently Improve the Robustness of CNNs
Authors:
Nikhil Kapoor,
Chun Yuan,
Jonas Löhdefink,
Roland Zimmermann,
Serin Varghese,
Fabian Hüger,
Nico Schmidt,
Peter Schlicht,
Tim Fingscheidt
Abstract:
Deep neural networks are often not robust to semantically-irrelevant changes in the input. In this work we address the issue of robustness of state-of-the-art deep convolutional neural networks (CNNs) against commonly occurring distortions in the input such as photometric changes, or the addition of blur and noise. These changes in the input are often accounted for during training in the form of d…
▽ More
Deep neural networks are often not robust to semantically-irrelevant changes in the input. In this work we address the issue of robustness of state-of-the-art deep convolutional neural networks (CNNs) against commonly occurring distortions in the input such as photometric changes, or the addition of blur and noise. These changes in the input are often accounted for during training in the form of data augmentation. We have two major contributions: First, we propose a new regularization loss called feature-map augmentation (FMA) loss which can be used during finetuning to make a model robust to several distortions in the input. Second, we propose a new combined augmentations (CA) finetuning strategy, that results in a single model that is robust to several augmentation types at the same time in a data-efficient manner. We use the CA strategy to improve an existing state-of-the-art method called stability training (ST). Using CA, on an image classification task with distorted images, we achieve an accuracy improvement of on average 8.94% with FMA and 8.86% with ST absolute on CIFAR-10 and 8.04% with FMA and 8.27% with ST absolute on ImageNet, compared to 1.98% and 2.12%, respectively, with the well known data augmentation method, while kee** the clean baseline performance.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
Evaluating the Contextual Integrity of Privacy Regulation: Parents' IoT Toy Privacy Norms Versus COPPA
Authors:
Noah Apthorpe,
Sarah Varghese,
Nick Feamster
Abstract:
Increased concern about data privacy has prompted new and updated data protection regulations worldwide. However, there has been no rigorous way to test whether the practices mandated by these regulations actually align with the privacy norms of affected populations. Here, we demonstrate that surveys based on the theory of contextual integrity provide a quantifiable and scalable method for measuri…
▽ More
Increased concern about data privacy has prompted new and updated data protection regulations worldwide. However, there has been no rigorous way to test whether the practices mandated by these regulations actually align with the privacy norms of affected populations. Here, we demonstrate that surveys based on the theory of contextual integrity provide a quantifiable and scalable method for measuring the conformity of specific regulatory provisions to privacy norms. We apply this method to the U.S. Children's Online Privacy Protection Act (COPPA), surveying 195 parents and providing the first data that COPPA's mandates generally align with parents' privacy expectations for Internet-connected "smart" children's toys. Nevertheless, variations in the acceptability of data collection across specific smart toys, information types, parent ages, and other conditions emphasize the importance of detailed contextual factors to privacy norms, which may not be adequately captured by COPPA.
△ Less
Submitted 12 March, 2019;
originally announced March 2019.
-
Heredity for generalized power domination
Authors:
Paul Dorbec,
Seethu Varghese,
Ambat Vijayakumar
Abstract:
In this paper, we study the behaviour of the generalized power domination number of a graph by small changes on the graph, namely edge and vertex deletion and edge contraction. We prove optimal bounds for $γ\_{p,k}(G-e)$, $γ\_{p,k}(G/e)$ and for $γ\_{p,k}(G-v)$ in terms of $γ\_{p,k}(G)$, and give examples for which these bounds are tight. We characterize all graphs for which…
▽ More
In this paper, we study the behaviour of the generalized power domination number of a graph by small changes on the graph, namely edge and vertex deletion and edge contraction. We prove optimal bounds for $γ\_{p,k}(G-e)$, $γ\_{p,k}(G/e)$ and for $γ\_{p,k}(G-v)$ in terms of $γ\_{p,k}(G)$, and give examples for which these bounds are tight. We characterize all graphs for which $γ\_{p,k}(G-e) = γ\_{p,k}(G)+1$ for any edge $e$. We also consider the behaviour of the propagation radius of graphs by similar modifications.
△ Less
Submitted 23 March, 2016;
originally announced March 2016.
-
Contrast Enhancement And Brightness Preservation Using Multi- Decomposition Histogram Equalization
Authors:
Sayali Nimkar,
Sanal Varghese,
Sucheta Shrivastava
Abstract:
Histogram Equalization (HE) has been an essential addition to the Image Enhancement world. Enhancement techniques like Classical Histogram Equalization (CHE), Adaptive Histogram Equalization (ADHE), Bi-Histogram Equalization (BHE) and Recursive Mean Separate Histogram Equalization (RMSHE) methods enhance contrast, however, brightness is not well preserved with these methods, which gives an unpleas…
▽ More
Histogram Equalization (HE) has been an essential addition to the Image Enhancement world. Enhancement techniques like Classical Histogram Equalization (CHE), Adaptive Histogram Equalization (ADHE), Bi-Histogram Equalization (BHE) and Recursive Mean Separate Histogram Equalization (RMSHE) methods enhance contrast, however, brightness is not well preserved with these methods, which gives an unpleasant look to the final image obtained. Thus, we introduce a novel technique Multi-Decomposition Histogram Equalization (MDHE) to eliminate the drawbacks of the earlier methods. In MDHE, we have decomposed the input sixty-four parts, applied CHE in each of the sub-images and then finally interpolated them in correct order. The final image after MDHE results in contrast enhanced and brightness preserved image compared to all other techniques mentioned above. We have calculated the various parameters like PSNR, SNR, RMSE, MSE, etc. for every technique. Our results are well supported by bar graphs, histograms and the parameter calculations at the end.
△ Less
Submitted 11 July, 2013;
originally announced July 2013.
-
Multidimensional User Data Model for Web Personalization
Authors:
Nithin K. Anil,
Sharath Basil Kurian,
Aby Abahai T,
Surekha Mariam Varghese
Abstract:
Personalization is being applied to great extend in many systems. This paper presents a multi-dimensional user data model and its application in web search. Online and Offline activities of the user are tracked for creating the user model. The main phases are identification of relevant documents and the representation of relevance and similarity of the documents. The concepts Keywords, Topics, URL…
▽ More
Personalization is being applied to great extend in many systems. This paper presents a multi-dimensional user data model and its application in web search. Online and Offline activities of the user are tracked for creating the user model. The main phases are identification of relevant documents and the representation of relevance and similarity of the documents. The concepts Keywords, Topics, URLs and clusters are used in the implementation. The algorithms for profiling, grading and clustering the concepts in the user model and algorithm for determining the personalized search results by re-ranking the results in a search bank are presented in this paper. Simple experiments for evaluation of the model and their results are described.
△ Less
Submitted 19 June, 2013;
originally announced June 2013.
-
Personalised product design using virtual interactive techniques
Authors:
Kurien Zacharia,
Eldo P. Elias,
Surekha Mariam Varghese
Abstract:
Use of Virtual Interactive Techniques for personalized product design is described in this paper. Usually products are designed and built by considering general usage patterns and Prototy** is used to mimic the static or working behaviour of an actual product before manufacturing the product. The user does not have any control on the design of the product. Personalized design postpones design to…
▽ More
Use of Virtual Interactive Techniques for personalized product design is described in this paper. Usually products are designed and built by considering general usage patterns and Prototy** is used to mimic the static or working behaviour of an actual product before manufacturing the product. The user does not have any control on the design of the product. Personalized design postpones design to a later stage. It allows for personalized selection of individual components by the user. This is implemented by displaying the individual components over a physical model constructed using Cardboard or Thermocol in the actual size and shape of the original product. The components of the equipment or product such as screen, buttons etc. are then projected using a projector connected to the computer into the physical model. Users can interact with the prototype like the original working equipment and they can select, shape, position the individual components displayed on the interaction panel using simple hand gestures. Computer Vision techniques as well as sound processing techniques are used to detect and recognize the user gestures captured using a web camera and microphone.
△ Less
Submitted 8 February, 2012;
originally announced February 2012.
-
Modelling Gesture Based Ubiquitous Applications
Authors:
Kurien Zacharia,
Eldo P. Elias,
Surekha Mariam Varghese
Abstract:
A cost effective, gesture based modelling technique called Virtual Interactive Prototy** (VIP) is described in this paper. Prototy** is implemented by projecting a virtual model of the equipment to be prototyped. Users can interact with the virtual model like the original working equipment. For capturing and tracking the user interactions with the model image and sound processing techniques ar…
▽ More
A cost effective, gesture based modelling technique called Virtual Interactive Prototy** (VIP) is described in this paper. Prototy** is implemented by projecting a virtual model of the equipment to be prototyped. Users can interact with the virtual model like the original working equipment. For capturing and tracking the user interactions with the model image and sound processing techniques are used. VIP is a flexible and interactive prototy** method that has much application in ubiquitous computing environments. Different commercial as well as socio-economic applications and extension to interactive advertising of VIP are also discussed.
△ Less
Submitted 9 December, 2011;
originally announced December 2011.
